Auxiliary sand control method, system and electronic equipment

Abstract

The present invention discloses an auxiliary sand control method, system and electronic device. A sand monitoring image is obtained. A background image is obtained. The sand monitoring image and the background image are input into a sand degree discrimination model to obtain sand concentration. Since it is difficult to extract features from sand, a background image and multiple training sand monitoring images in a training set are used to input into a training sand degree discrimination model. This enables better detection of color information in the image, and extracts features through a convolutional network. Overall discrimination is performed through grayscale difference. The sand feature map is then divided into multiple local sand feature vectors, and after performing difference calculation with the local background feature vector, the vectors are input into a local discriminant neural network to predict the local sand state. The sand concentration can be judged more accurately by using the information of each local area. The accuracy of the sand concentration detection is determined by finding the sand concentration with the most similarity values.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an auxiliary sand control method, system and electronic equipment. Background Art

[0002] At present, as environmental protection is in full swing, sand control is also a part that needs attention. However, since sand has no specific shape and the degree of aggregation leads to different degrees of color, it is not easy to detect or the detection level is not accurate enough. Summary of the Invention

[0003] The purpose of the present invention is to provide an auxiliary sand control method, system and electronic equipment to solve the above-mentioned problems existing in the prior art.

[0004] In a first aspect, an embodiment of the present invention provides an auxiliary sand control method, comprising:

[0005] Obtaining wind and sand monitoring images; the wind and sand monitoring images represent images of wind and sand captured at different locations during the same period;

[0006] Obtaining a background image; the background image represents an image captured by a fixed monitoring system when there is no wind and sand;

[0007] Inputting the wind and sand monitoring image and the background image into a wind and sand degree discrimination model to obtain wind and sand concentration; the value in the wind and sand concentration represents the degree of wind and sand in the wind and sand monitoring image;

[0008] The wind and sand degree discrimination model includes a background convolutional network, a wind and sand convolutional network, a wind and sand overall feature discrimination structure and a local discrimination neural network.

[0009] Optional training methods for the sandstorm severity discrimination model include:

[0010] A training set is obtained; the training set includes a training background image, a plurality of training wind and sand image groups, and annotated data; the annotated data is manually annotated wind and sand concentration; the annotated data is an annotated wind and sand state vector;

[0011] Inputting the training background image and the training wind sand image into a wind sand degree discrimination model to obtain training wind sand concentration information; the training wind sand concentration information includes a local wind sand state vector and a training wind sand concentration value;

[0012] Obtaining a local wind and sand state loss value based on the local wind and sand state vector and the marked local wind and sand state vector;

[0013] Obtaining a wind sand concentration loss value based on the training wind sand concentration value and the labeled wind sand state vector;

[0014] Obtain a total loss value; the total loss value is the sum of the local wind and sand state loss value and the wind and sand concentration loss value;

[0015] Obtaining the current training iteration number of the wind and sand degree discrimination model and the preset maximum training iteration number of the wind and sand degree discrimination model;

[0016] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sandstorm degree discrimination model.

[0017] Optionally, inputting the training background image and the training wind and sand image into a wind and sand degree discrimination model to obtain training wind and sand concentration information includes:

[0018] Based on the wind and sand monitoring image, a training HSV wind and sand monitoring image is obtained; the training HSV wind and sand monitoring image represents an image after the wind and sand monitoring image is converted from RGB to HSV state;

[0019] Based on the background image, a training HSV background image is obtained; the training HSV background image represents an image after the background image is converted from RGB to HSV state;

[0020] Input the training HSV background image into the background convolutional network, extract the background image features, and obtain a background feature map;

[0021] Input the training HSV wind and sand monitoring image into the wind and sand convolutional network, extract the wind and sand image features, and obtain a wind and sand feature map;

[0022] Based on the background feature map and the wind and sand feature map, training wind and sand concentration information is obtained.

[0023] Optionally, obtaining training wind and sand concentration information based on the background feature map and the wind and sand feature map includes:

[0024] Based on the background feature map and the wind and sand feature map, a wind and sand overall discrimination value is obtained through a wind and sand overall feature discrimination structure; the wind and sand overall discrimination value of 1 indicates the presence of wind and sand; and the wind and sand overall discrimination value of 0 indicates the absence of wind and sand.

[0025] Obtaining a plurality of local wind-sand feature vectors; wherein the local wind-sand feature vectors are feature vectors obtained by dividing the wind-sand feature map into a plurality of 1x1 feature maps;

[0026] Obtaining a plurality of local background feature vectors; wherein the local background feature vector is a feature vector obtained by dividing the background feature map into a plurality of 1x1 feature maps;

[0027] Inputting the local wind sand feature vector and the local background feature vector into a local discriminant neural network to obtain a local wind sand state vector; the local wind sand state vector represents the presence and degree of wind sand in the area;

[0028] By passing multiple local wind-sand feature vectors and corresponding local background feature vectors multiple times, multiple local wind-sand state vectors are obtained;

[0029] A training wind sand concentration value is obtained by combining a plurality of local wind sand state vectors and the wind sand overall discrimination value.

[0030] Optionally, obtaining the wind and sand overall discrimination value based on the background feature map and the wind and sand feature map through a wind and sand overall feature discrimination structure includes:

[0031] Based on the background image, obtaining a background grayscale image;

[0032] Based on the wind and sand monitoring image, a wind and sand grayscale image is obtained;

[0033] Based on the background grayscale image and the wind-sand grayscale image, a wind-sand grayscale difference image is obtained;

[0034] Based on the wind-sand grayscale difference image, absolute value addition is performed to obtain the wind-sand grayscale difference;

[0035] If the wind and sand grayscale difference is greater than the wind and sand grayscale threshold, the wind and sand overall discrimination value is set to 1;

[0036] If the wind and sand grayscale difference is less than the wind and sand grayscale threshold, the wind and sand overall discrimination value is set to 0.

[0037] Optionally, inputting the local wind-sand feature vector and the local background feature vector into a local discriminant neural network to obtain a local wind-sand state vector includes:

[0038] Based on the local wind-sand feature vector and the local background feature vector, a local difference feature vector is obtained; the local difference feature vector is a difference between the two features;

[0039] Based on the local difference eigenvector, obtaining a local non-negative difference eigenvector;

[0040] The local non-negative difference feature vector is input into a local discriminant neural network to obtain a local wind and sand state vector; the vector length of the local wind and sand state vector is 11; the first vector value of the local wind and sand state vector is 1, indicating the presence of wind and sand, and 0, indicating the absence of wind and sand; the last 10 bits of the vector value in the local wind and sand state vector indicate the degree of wind and sand presence.

[0041] Optionally, the step of obtaining a training wind sand concentration value by combining multiple local wind sand state vectors and the wind sand overall discrimination value includes:

[0042] Based on the multiple local wind sand state vectors, a local wind sand state matrix is ​​obtained; the local wind sand state matrix represents the local wind sand state matrix formed by combining the multiple local wind sand state vectors in a segmentation order;

[0043] Based on the local wind and sand state matrix, obtaining a first local wind and sand state value;

[0044] Obtaining a first overall similarity value based on the first local wind and sand state value and the wind and sand overall discrimination value;

[0045] obtaining a first surrounding similarity value based on the first local wind and sand state value and the local wind and sand state matrix; wherein the first surrounding similarity value represents an average of similarity values ​​between the first local wind and sand state value and the surrounding state value;

[0046] By repeatedly obtaining multiple local sand state values, multiple overall similarity values ​​and multiple surrounding similarity values ​​corresponding to ;

[0047] A training wind sand concentration value is obtained based on the multiple overall similarity values ​​and the multiple surrounding similarity values.

[0048] Optionally, obtaining a training wind sand concentration value based on the multiple overall similarity values ​​and the multiple surrounding similarity values ​​includes:

[0049] Get the overall similarity threshold;

[0050] Based on the multiple overall similarity values ​​and the overall similarity threshold, a plurality of overall threshold wind sand concentrations are obtained; the overall threshold wind sand concentration is the wind sand concentration whose overall similarity value is less than the overall similarity threshold;

[0051] Get the local similarity threshold;

[0052] Based on the multiple surrounding similarity values ​​and the local similarity threshold, a multiple local threshold wind sand concentration is obtained; the local threshold wind sand concentration is the wind sand concentration when the surrounding similarity value is less than the local similarity threshold;

[0053] Based on the multiple overall threshold wind sand concentrations and the multiple local threshold wind sand concentrations, a training wind sand concentration is obtained.

[0054] In a second aspect, an embodiment of the present invention provides an auxiliary sand control system, comprising:

[0055] Acquisition module: obtains wind and sand monitoring images; the wind and sand monitoring images represent images with wind and sand captured at different locations during the same period; obtains background images; the background images represent images without wind and sand captured by fixed monitoring;

[0056] A discrimination module: inputs the wind and sand monitoring image and the background image into a wind and sand degree discrimination model to obtain a wind and sand concentration; the value of the wind and sand concentration indicates the degree of wind and sand in the wind and sand monitoring image;

[0057] The wind and sand degree discrimination model includes a background convolutional network, a wind and sand convolutional network, a wind and sand overall feature discrimination structure and a local discrimination neural network.

[0058] Optionally, the training method of the wind and sand severity discrimination model includes:

[0059] A training set is obtained; the training set includes a training background image, a plurality of training wind and sand image groups, and annotated data; the annotated data is manually annotated wind and sand concentration; the annotated data is an annotated wind and sand state vector;

[0060] Inputting the training background image and the training wind sand image into a wind sand degree discrimination model to obtain training wind sand concentration information; the training wind sand concentration information includes a local wind sand state vector and a training wind sand concentration value;

[0061] Obtaining a local wind and sand state loss value based on the local wind and sand state vector and the marked local wind and sand state vector;

[0062] Obtaining a wind sand concentration loss value based on the training wind sand concentration value and the labeled wind sand state vector;

[0063] Obtain a total loss value; the total loss value is the sum of the local wind and sand state loss value and the wind and sand concentration loss value;

[0064] Obtaining the current training iteration number of the wind and sand degree discrimination model and the preset maximum training iteration number of the wind and sand degree discrimination model;

[0065] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sandstorm degree discrimination model.

[0066] In a third aspect, an embodiment of the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the program.

[0067] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0068] Embodiments of the present invention also provide a method, system, and electronic device for assisting with sand control. The method includes: obtaining a sand monitoring image. The sand monitoring image represents an image of sand present captured at different locations during the same time period. Obtaining a background image. The background image represents an image captured by a fixed monitoring system without sand present. Inputting the sand monitoring image and the background image into a sand intensity discrimination model to obtain sand concentration. The value in the sand concentration represents the degree of sand present in the sand monitoring image. The sand intensity discrimination model includes a background convolutional network, a sand convolutional network, a sand overall feature discrimination structure, and a local discriminant neural network.

[0069] Because it's difficult to extract features from wind and sand, a wind and sand severity discrimination model is trained to determine wind and sand concentration using wind and sand monitoring images. The training model is fed with background images from the training set and multiple training wind and sand monitoring images. The background and wind and sand monitoring images are converted from RGB format to HSV format, enabling better detection of color information within the images and facilitating the use of color to determine the presence of wind and sand within the images. A convolutional network is used to extract features from the wind and sand monitoring and background images. The resulting background and wind and sand feature maps are first subjected to a global discrimination using grayscale difference to obtain a global wind and sand discrimination value. The wind and sand feature map is then divided into multiple local wind and sand feature vectors. These vectors are then subtracted from the corresponding local background feature vectors from the background feature map. These vectors are then fed into a local discriminant neural network to predict the local wind and sand state, determining the wind and sand conditions in the local area. Compared to global discrimination, utilizing information from each local area to determine wind and sand concentration allows for more accurate judgments. At the same time, multiple local wind and sand state vectors and the overall wind and sand discrimination value are combined. Due to the characteristics of wind and sand, the captured wind and sand are almost evenly distributed in the image. By finding the wind and sand concentration with the most similar values, the training wind and sand concentration value is obtained. The accuracy of wind and sand concentration detection can be judged by the degree of similarity with the surrounding area, and training data can be obtained more accurately during the training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of an auxiliary sand control method provided by an embodiment of the present invention.

[0071] Figure 2 It is a structural schematic diagram of the wind and sand degree discrimination model in the auxiliary wind and sand control system provided by an embodiment of the present invention.

[0072] Figure 3 This is a flow chart of a method for obtaining training wind sand concentration values ​​from a background feature map and a wind sand feature map in an auxiliary wind sand control method provided by an embodiment of the present invention.

[0073] Figure 4It is a block diagram of an electronic device provided by an embodiment of the present invention.

[0074] Labels in the figure: bus 500; receiver 501; processor 502; transmitter 503; memory 504; bus interface 505. DETAILED DESCRIPTION

[0075] The present invention will be described in detail below with reference to the accompanying drawings.

[0076] Example 1

[0077] like Figure 1 As shown, an embodiment of the present invention provides an auxiliary sand control method, the method comprising:

[0078] S101: Obtaining wind and sand monitoring images; the wind and sand monitoring images are images with wind and sand captured at different locations during the same period of time.

[0079] Among them, the movement path and speed of the wind and sand can be determined by images of wind and sand taken at different locations at the same time.

[0080] Among them, since wind sand is in dust state and has almost no shape during movement, it is impossible to judge wind sand by its movement state. The existence and degree of wind sand can only be judged by its characteristics.

[0081] S102: Obtain a background image; the background image represents an image captured by fixed monitoring without the presence of wind and sand.

[0082] S103: Input the wind and sand monitoring image and the background image into a wind and sand degree discrimination model to obtain wind and sand concentration. The value of the wind and sand concentration represents the degree of wind and sand in the wind and sand monitoring image.

[0083] Wherein, a sand concentration greater than the sand concentration threshold indicates the presence of sand, and a sand concentration less than the sand concentration threshold indicates the absence of sand.

[0084] In this embodiment, the sand concentration threshold is 30%.

[0085] The wind and sand degree discrimination model includes a background convolutional network, a wind and sand convolutional network, a wind and sand overall feature discrimination structure and a local discrimination neural network.

[0086] The above method is mainly used to determine whether there is wind and sand. The characteristics of wind and sand are difficult to identify, so a neural network is used to obtain a model that can detect wind and sand through a large amount of training.

[0087] Optionally, the training method of the wind and sand severity discrimination model includes:

[0088] A training set is obtained; the training set includes a training background image, a plurality of training wind and sand image groups, and annotated data. The annotated data is manually annotated wind and sand concentration. The annotated data is annotated local wind and sand state vector.

[0089] In this embodiment, the vector value for sand concentration includes vector values ​​indicating the presence and degree of sand. The vector indicating the presence of sand has a length of 1, with a value of 1 indicating the presence of sand and a value of 0 indicating the absence of sand. The vector indicating the degree of sand also has a length of 10, representing ten states: 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, and 100%, indicating the degree of sand in the sand monitoring image.

[0090] The training background image and the training wind sand image are input into a wind sand degree discrimination model to obtain training wind sand concentration information, which includes a local wind sand state vector and a training wind sand concentration value.

[0091] A local wind and sand state loss value is obtained based on the local wind and sand state vector and the marked local wind and sand state vector.

[0092] The local wind and sand state loss value is calculated by using a cross entropy loss function on the local wind and sand state vector and the labeled local wind and sand state vector.

[0093] A wind sand concentration loss value is obtained based on the training wind sand concentration value and the labeled wind sand state vector.

[0094] The wind sand concentration loss value is calculated using a cross entropy loss function using the training wind sand concentration value and the labeled wind sand state vector.

[0095] Obtain a total loss value; the total loss value is the sum of the local wind and sand state loss value and the wind and sand concentration loss value.

[0096] The current training iteration number of the wind and sand degree discrimination model and the preset maximum iteration number of the wind and sand degree discrimination model training are obtained.

[0097] The maximum number of iterations of the wind and sand degree discrimination model training preset in this embodiment is 1200 times.

[0098] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sandstorm degree discrimination model.

[0099] The above method is difficult to use because windblown sand typically exists outdoors. If it is present, the image captured will contain the same concentration of sand throughout the entire frame, making it difficult to determine its shape and location. Therefore, a convolutional network trained with large amounts of data can be used to identify the presence and degree of windblown sand using features that are difficult to directly identify.

[0100] Optionally, inputting the training background image and the training wind and sand image into a wind and sand degree discrimination model to obtain training wind and sand concentration information includes:

[0101] Based on the wind and sand monitoring image, a training HSV wind and sand monitoring image is obtained; the training HSV wind and sand monitoring image represents an image after the wind and sand monitoring image is converted from RGB to HSV state.

[0102] Based on the background image, a training HSV background image is obtained; the training HSV background image represents an image after the background image is converted from RGB to HSV state.

[0103] The training HSV background image is input into the background convolutional network to extract background image features to obtain a background feature map.

[0104] The training HSV wind and sand monitoring image is input into the wind and sand convolutional network to extract wind and sand image features to obtain a wind and sand feature map.

[0105] Based on the background feature map and the wind and sand feature map, training wind and sand concentration information is obtained.

[0106] The schematic diagram of the wind and sand degree discrimination model is as follows: Figure 2 shown.

[0107] By converting the image from RBG to HSV using the above method, the color information in the image can be better detected, making it easier to use the color in the image to judge the presence of wind and sand in the image. At the same time, the convolutional network is used to extract the features of the background image and the wind and sand monitoring image, and the differences between the two are used to jointly determine the presence of wind and sand.

[0108] Optionally, obtaining training wind and sand concentration information based on the background feature map and the wind and sand feature map includes:

[0109] Based on the background feature map and the wind and sand feature map, a wind and sand overall discrimination value is obtained through a wind and sand overall feature discrimination structure; the wind and sand overall discrimination value of 1 indicates the presence of wind and sand; the wind and sand overall discrimination value of 0 indicates the absence of wind and sand.

[0110] A plurality of local wind-sand feature vectors are obtained; the local wind-sand feature vectors are feature vectors obtained by dividing the wind-sand feature map into a plurality of 1x1 feature maps.

[0111] A plurality of local background feature vectors are obtained; the local background feature vectors are feature vectors obtained by dividing the background feature map into a plurality of 1x1 feature maps.

[0112] The local wind and sand feature vector and the local background feature vector are input into a local discriminant neural network to obtain a local wind and sand state vector; the local wind and sand state vector represents the existence and degree of wind and sand in the area.

[0113] By passing multiple local wind-sand feature vectors and corresponding local background feature vectors multiple times, multiple local wind-sand state vectors are obtained.

[0114] A training wind sand concentration value is obtained by combining a plurality of local wind sand state vectors and the wind sand overall discrimination value.

[0115] The schematic diagram of obtaining the training wind sand concentration value obtained from the background feature map and the wind sand feature map is as follows: Figure 3 shown.

[0116] Through the above method, the wind and sand characteristic map is used to make separate judgments, so that the wind and sand conditions in each area can be accurately obtained, which is less likely to cause errors than overall judgment.

[0117] Optionally, obtaining the wind and sand overall discrimination value based on the background feature map and the wind and sand feature map through a wind and sand overall feature discrimination structure includes:

[0118] Based on the background image, a background grayscale image is obtained.

[0119] A wind and sand grayscale image is obtained based on the wind and sand monitoring image.

[0120] Based on the background grayscale image and the wind-sand grayscale image, a wind-sand grayscale difference image is obtained.

[0121] Based on the wind-sand grayscale difference image, absolute value addition is performed to obtain the wind-sand grayscale difference.

[0122] If the wind and sand grayscale difference is greater than the wind and sand grayscale threshold, the wind and sand overall discrimination value is set to 1.

[0123] If the wind and sand grayscale difference is less than the wind and sand grayscale threshold, the wind and sand overall discrimination value is set to 0.

[0124] Through the above method, the overall image difference is obtained by the difference in grayscale values ​​between the background image and the wind and sand monitoring image. At the same time, the influencing factors of the overall judgment are considered in the subsequent judgment, which can more accurately obtain the predicted wind and sand concentration value.

[0125] Optionally, inputting the local wind-sand feature vector and the local background feature vector into a local discriminant neural network to obtain a local wind-sand state vector includes:

[0126] Based on the local wind-sand feature vector and the local background feature vector, a local difference feature vector is obtained; the local difference feature vector is a difference between the two features;

[0127] Based on the local difference eigenvector, a local non-negative difference eigenvector is obtained.

[0128] The local difference eigenvector is subjected to the ReLU function to obtain a local non-negative difference eigenvector.

[0129] The local non-negative difference feature vector is input into a local discriminant neural network to obtain a local sandstorm state vector. The vector length of the local sandstorm state vector is 11. The first digit of the vector value in the local sandstorm state vector indicates the presence of sandstorms, while the first digit indicates the absence of sandstorms. The last 10 digits of the vector value in the local sandstorm state vector indicate the degree of sandstorm presence.

[0130] The last 10 bits of the vector value in the local wind and sand state vector represent ten states of 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, and 100% of the wind and sand in the monitoring image.

[0131] The above method utilizes local information to more accurately determine sand concentration. The sand state output is not a 0 or 1 judgment, but rather a holistic judgment that includes the degree of sand presence. After obtaining some characteristic conditions from the feature map, the sand feature map is processed.

[0132] Optionally, the step of obtaining a training wind sand concentration value by combining multiple local wind sand state vectors and the wind sand overall discrimination value includes:

[0133] Based on multiple local wind sand state vectors, a local wind sand state matrix is ​​obtained; the local wind sand state matrix represents the combination of the multiple local wind sand state vectors according to the segmentation order into the local wind sand state matrix.

[0134] The size of the local wind and sand state matrix is ​​fixed according to the setting of the convolutional network. In this embodiment, the size of the local wind and sand state matrix is ​​7x7.

[0135] Based on the local wind and sand state matrix, a first local wind and sand state value is obtained.

[0136] A first overall similarity value is obtained based on the first local wind and sand state value and the wind and sand overall discrimination value.

[0137] The first overall similarity value is obtained by using the Euclidean distance algorithm, and the first overall similarity value indicates the degree of similarity between the first local wind and sand state value and the wind and sand overall discrimination value.

[0138] A first surrounding similarity value is obtained based on the first local wind and sand state value and the local wind and sand state matrix. The first surrounding similarity value represents an average value of similarity values ​​between the first local wind and sand state value and the surrounding state value.

[0139] The surrounding similarity value is the average of the similarity values ​​calculated using the Euclidean distance algorithm between a value and the four surrounding values ​​above, below, left, and right. However, due to different locations, for example, the value in the upper left corner of the local sandstorm state matrix is ​​surrounded by only two values ​​to the right and below, so only two similarity values ​​are calculated.

[0140] By obtaining multiple local wind and sand state values ​​multiple times, and corresponding multiple overall similarity values ​​and multiple surrounding similarity values; comparing the overall similarity value with the overall similarity threshold, and comparing the surrounding similarity value with the local similarity threshold, wind and sand state discrimination information is obtained.

[0141] The calculation method of the plurality of overall similarity values ​​is the same as that of the first overall similarity value, and the calculation method of the plurality of surrounding similarity values ​​is the same as that of the first surrounding similarity value.

[0142] Through the above method, due to the characteristics of wind and sand, the captured wind and sand are almost evenly distributed in the image. Therefore, the accuracy of wind and sand concentration detection is judged by the degree of similarity with the surrounding area, so that training data can be obtained more accurately during the training process.

[0143] Optionally, obtaining a training wind sand concentration value based on the multiple overall similarity values ​​and the multiple surrounding similarity values ​​includes:

[0144] Get the overall similarity threshold;

[0145] Based on the multiple overall similarity values ​​and the overall similarity threshold, a plurality of overall threshold wind sand concentrations are obtained; the overall threshold wind sand concentration is the wind sand concentration whose overall similarity value is less than the overall similarity threshold;

[0146] Get the local similarity threshold;

[0147] Based on the multiple surrounding similarity values ​​and the local similarity threshold, a multiple local threshold wind sand concentration is obtained; the local threshold wind sand concentration is the wind sand concentration when the surrounding similarity value is less than the local similarity threshold;

[0148] Obtaining a training wind sand concentration value based on the multiple global threshold wind sand concentrations and the multiple local threshold wind sand concentrations;

[0149] Through the above method, the wind and sand with the greatest similarity in each region of the entire wind and sand monitoring image is obtained as the training wind and sand concentration value detected during the training process. This is the result determined based on the characteristics of the wind and sand.

[0150] Because it's difficult to extract features from wind and sand using the above method, a wind and sand severity discrimination model is trained to determine wind and sand concentration from wind and sand monitoring images. The training model is fed with a background image from the training set and multiple training wind and sand monitoring images. The background image and wind and sand monitoring images are converted from RGB format to HSV format, enabling better detection of color information within the images and facilitating the use of color to determine the presence of wind and sand within the images. A convolutional network is used to extract features from the wind and sand monitoring and background images. The resulting background feature map and wind and sand feature map are first subjected to a global discrimination using grayscale difference to obtain a global wind and sand discrimination value. The wind and sand feature map is then divided into multiple local wind and sand feature vectors. These features are then subtracted from the corresponding local background feature vectors from the background feature map. These features are then fed into a local discriminant neural network to predict the local wind and sand status, determining the wind and sand conditions in the local area. Compared to global discrimination, this method utilizes information from each local area to more accurately determine wind and sand concentration. At the same time, multiple local wind and sand state vectors and the overall wind and sand discrimination value are combined. Due to the characteristics of wind and sand, the captured wind and sand are almost evenly distributed in the image. By finding the wind and sand concentration with the most similar values, the training wind and sand concentration value is obtained, so that the accuracy of wind and sand concentration detection can be judged by the degree of similarity with the surrounding area.

[0151] Example 2

[0152] Based on the above-mentioned auxiliary sand control method, an embodiment of the present invention further provides an auxiliary sand control system, which includes an acquisition module and a discrimination module.

[0153] The acquisition module is used to obtain wind and sand monitoring images and background images. The wind and sand monitoring images represent images captured at different locations and at the same time period with wind and sand present. The background images represent images captured by fixed monitoring without wind and sand present.

[0154] The discrimination module is used to input the wind and sand monitoring image and the background image into a wind and sand degree discrimination model to obtain wind and sand concentration; the value of the wind and sand concentration represents the degree of wind and sand in the wind and sand monitoring image;

[0155] The wind and sand degree discrimination model includes a background convolutional network, a wind and sand convolutional network, a wind and sand overall feature discrimination structure and a local discrimination neural network.

[0156] Optionally, the training method of the wind and sand severity discrimination model includes:

[0157] A training set is obtained; the training set includes a training background image, a plurality of training wind and sand image groups, and annotated data; the annotated data is manually annotated wind and sand concentration; the annotated data is an annotated wind and sand state vector;

[0158] Inputting the training background image and the training wind sand image into a wind sand degree discrimination model to obtain training wind sand concentration information; the training wind sand concentration information includes a local wind sand state vector and a training wind sand concentration value;

[0159] Obtaining a local wind and sand state loss value based on the local wind and sand state vector and the marked local wind and sand state vector;

[0160] Obtaining a wind sand concentration loss value based on the training wind sand concentration value and the labeled wind sand state vector;

[0161] Obtain a total loss value; the total loss value is the sum of the local wind and sand state loss value and the wind and sand concentration loss value;

[0162] Obtaining the current training iteration number of the wind and sand degree discrimination model and the preset maximum training iteration number of the wind and sand degree discrimination model;

[0163] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sandstorm degree discrimination model.

[0164] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0165] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, it includes a memory 504, a processor 502 and a computer program stored in the memory 504 and executable on the processor 502. When the processor 502 executes the program, the steps of any one of the aforementioned auxiliary sand control methods are implemented.

[0166] Among them, Figure 4In the embodiment of the present invention, a bus architecture (represented by bus 500) is shown. Bus 500 may include any number of interconnected buses and bridges, and bus 500 links various circuits including one or more processors represented by processor 502 and memory represented by memory 504. Bus 500 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 505 provides an interface between bus 500 and receiver 501 and transmitter 503. Receiver 501 and transmitter 503 may be the same component, namely a transceiver, which provides a unit for communicating with various other devices over a transmission medium. Processor 502 is responsible for managing bus 500 and general processing, while memory 504 may be used to store data used by processor 502 when performing operations.

[0167] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any one of the aforementioned auxiliary sand control methods and the aforementioned data involved.

[0168] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general-purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing this type of system. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0169] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0170] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0171] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0172] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0173] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to an embodiment of the present invention. The present invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing a part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0174] It should be noted that the above embodiments illustrate rather than limit the invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

Claims

1. An auxiliary sand control method, characterized in that: include: Obtain wind and sand monitoring images; The wind and sand monitoring images refer to images with wind and sand captured at different locations during the same period of time; Get the background image; The background image represents an image captured by fixed monitoring without the presence of wind and sand; Inputting the wind and sand monitoring image and the background image into a wind and sand degree discrimination model to obtain wind and sand concentration; the value in the wind and sand concentration represents the degree of wind and sand in the wind and sand monitoring image; The wind and sand degree discrimination model includes a background convolutional network, a wind and sand convolutional network, a wind and sand overall feature discrimination structure and a local discrimination neural network; The training method of the wind and sand severity discrimination model includes: A training set is obtained; the training set includes a training background image, a plurality of training wind and sand image groups, and annotated data; the annotated data is manually annotated wind and sand concentration; the annotated data is an annotated wind and sand state vector; Inputting the training background image and the training wind sand image into a wind sand degree discrimination model to obtain training wind sand concentration information; the training wind sand concentration information includes a local wind sand state vector and a training wind sand concentration value; Obtaining a local wind and sand state loss value based on the local wind and sand state vector and the marked local wind and sand state vector; Obtaining a wind sand concentration loss value based on the training wind sand concentration value and the labeled wind sand state vector; Obtain a total loss value; the total loss value is the sum of the local wind and sand state loss value and the wind and sand concentration loss value; Obtaining the current training iteration number of the wind and sand degree discrimination model and the preset maximum training iteration number of the wind and sand degree discrimination model; When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sandstorm degree discrimination model; The step of inputting the training background image and the training wind and sand image into the wind and sand degree discrimination model to obtain training wind and sand concentration information includes: Based on the wind and sand monitoring image, a training HSV wind and sand monitoring image is obtained; the training HSV wind and sand monitoring image represents an image after the wind and sand monitoring image is converted from RGB to HSV state; Based on the background image, a training HSV background image is obtained; the training HSV background image represents an image after the background image is converted from RGB to HSV state; Input the training HSV background image into the background convolutional network, extract the background image features, and obtain a background feature map; Input the training HSV wind and sand monitoring image into the wind and sand convolutional network, extract the wind and sand image features, and obtain a wind and sand feature map; Based on the background feature map and the wind and sand feature map, obtaining training wind and sand concentration information; The obtaining of training wind and sand concentration information based on the background feature map and the wind and sand feature map includes: Based on the background feature map and the wind and sand feature map, a wind and sand overall discrimination value is obtained through a wind and sand overall feature discrimination structure; the wind and sand overall discrimination value of 1 indicates the presence of wind and sand; and the wind and sand overall discrimination value of 0 indicates the absence of wind and sand. Obtaining a plurality of local wind-sand feature vectors; wherein the local wind-sand feature vectors are feature vectors obtained by dividing the wind-sand feature map into a plurality of 1x1 feature maps; Obtaining a plurality of local background feature vectors; wherein the local background feature vector is a feature vector obtained by dividing the background feature map into a plurality of 1x1 feature maps; Inputting the local wind sand feature vector and the local background feature vector into a local discriminant neural network to obtain a local wind sand state vector; the local wind sand state vector represents the presence and degree of wind sand in the area; By passing multiple local wind-sand feature vectors and corresponding local background feature vectors multiple times, multiple local wind-sand state vectors are obtained; A training wind sand concentration value is obtained by combining a plurality of local wind sand state vectors and the wind sand overall discrimination value.

2. The auxiliary sand control method according to claim 1, characterized in that: The step of obtaining a wind and sand overall discrimination value based on the background feature map and the wind and sand feature map by using a wind and sand overall feature discrimination structure includes: Based on the background image, obtaining a background grayscale image; Based on the wind and sand monitoring image, a wind and sand grayscale image is obtained; Based on the background grayscale image and the wind-sand grayscale image, a wind-sand grayscale difference image is obtained; Based on the wind-sand grayscale difference image, absolute value addition is performed to obtain the wind-sand grayscale difference; If the wind and sand grayscale difference is greater than the wind and sand grayscale threshold, the wind and sand overall discrimination value is set to 1; If the wind and sand grayscale difference is less than the wind and sand grayscale threshold, the wind and sand overall discrimination value is set to 0.

3. The auxiliary sand control method according to claim 1, characterized in that: The local wind and sand feature vector and the local background feature vector are input into a local discriminant neural network to obtain a local wind and sand state vector, including: Based on the local wind-sand feature vector and the local background feature vector, a local difference feature vector is obtained; the local difference feature vector is a difference between the two features; Based on the local difference eigenvector, obtaining a local non-negative difference eigenvector; The local non-negative difference feature vector is input into a local discriminant neural network to obtain a local wind and sand state vector; the vector length of the local wind and sand state vector is 11; the first vector value of the local wind and sand state vector is 1, indicating the presence of wind and sand, and 0, indicating the absence of wind and sand; the last 10 bits of the vector value in the local wind and sand state vector indicate the degree of wind and sand presence.

4. The auxiliary sand control method according to claim 1, characterized in that: The step of obtaining a training wind sand concentration value by combining a plurality of local wind sand state vectors and the wind sand overall discrimination value includes: Based on the multiple local wind sand state vectors, a local wind sand state matrix is ​​obtained; the local wind sand state matrix represents the local wind sand state matrix formed by combining the multiple local wind sand state vectors in a segmentation order; Based on the local wind and sand state matrix, obtaining a first local wind and sand state value; Obtaining a first overall similarity value based on the first local wind and sand state value and the wind and sand overall discrimination value; obtaining a first surrounding similarity value based on the first local wind and sand state value and the local wind and sand state matrix; wherein the first surrounding similarity value represents an average of similarity values ​​between the first local wind and sand state value and the surrounding state value; A plurality of local wind and sand state values ​​are obtained multiple times, and a plurality of overall similarity values ​​and a plurality of surrounding similarity values ​​corresponding to the values ​​are obtained; and a training wind and sand concentration value is obtained based on the plurality of overall similarity values ​​and the plurality of surrounding similarity values.

5. An auxiliary sand control system, characterized in that: include: Acquisition module: obtains wind and sand monitoring images; the wind and sand monitoring images represent images with wind and sand captured at different locations during the same period; Get the background image; The background image represents an image captured by fixed monitoring without the presence of wind and sand; A discrimination module: inputs the wind and sand monitoring image and the background image into a wind and sand degree discrimination model to obtain a wind and sand concentration; the value of the wind and sand concentration indicates the degree of wind and sand in the wind and sand monitoring image; The wind and sand degree discrimination model includes a background convolutional network, a wind and sand convolutional network, a wind and sand overall feature discrimination structure and a local discrimination neural network; The training method of the wind and sand severity discrimination model includes: A training set is obtained; the training set includes a training background image, a plurality of training wind and sand image groups, and annotated data; the annotated data is manually annotated wind and sand concentration; the annotated data is an annotated wind and sand state vector; Inputting the training background image and the training wind sand image into a wind sand degree discrimination model to obtain training wind sand concentration information; the training wind sand concentration information includes a local wind sand state vector and a training wind sand concentration value; Obtaining a local wind and sand state loss value based on the local wind and sand state vector and the marked local wind and sand state vector; Obtaining a wind sand concentration loss value based on the training wind sand concentration value and the labeled wind sand state vector; Obtain a total loss value; the total loss value is the sum of the local wind and sand state loss value and the wind and sand concentration loss value; Obtaining the current training iteration number of the wind and sand degree discrimination model and the preset maximum training iteration number of the wind and sand degree discrimination model; When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained sandstorm degree discrimination model; The step of inputting the training background image and the training wind and sand image into the wind and sand degree discrimination model to obtain training wind and sand concentration information includes: Based on the wind and sand monitoring image, a training HSV wind and sand monitoring image is obtained; the training HSV wind and sand monitoring image represents an image after the wind and sand monitoring image is converted from RGB to HSV state; Based on the background image, a training HSV background image is obtained; the training HSV background image represents an image after the background image is converted from RGB to HSV state; Input the training HSV background image into the background convolutional network, extract the background image features, and obtain a background feature map; Input the training HSV wind and sand monitoring image into the wind and sand convolutional network, extract the wind and sand image features, and obtain a wind and sand feature map; Based on the background feature map and the wind and sand feature map, obtaining training wind and sand concentration information; The obtaining of training wind and sand concentration information based on the background feature map and the wind and sand feature map includes: Based on the background feature map and the wind and sand feature map, a wind and sand overall discrimination value is obtained through a wind and sand overall feature discrimination structure; the wind and sand overall discrimination value of 1 indicates the presence of wind and sand; and the wind and sand overall discrimination value of 0 indicates the absence of wind and sand. Obtaining a plurality of local wind-sand feature vectors; wherein the local wind-sand feature vectors are feature vectors obtained by dividing the wind-sand feature map into a plurality of 1x1 feature maps; Obtaining a plurality of local background feature vectors; wherein the local background feature vector is a feature vector obtained by dividing the background feature map into a plurality of 1x1 feature maps; Inputting the local wind sand feature vector and the local background feature vector into a local discriminant neural network to obtain a local wind sand state vector; the local wind sand state vector represents the presence and degree of wind sand in the area; By passing multiple local wind-sand feature vectors and corresponding local background feature vectors multiple times, multiple local wind-sand state vectors are obtained; A training wind sand concentration value is obtained by combining a plurality of local wind sand state vectors and the wind sand overall discrimination value.

6. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the method according to any one of claims 1 to 4 are implemented when the processor executes the program.

Images