Rural environment identification method, device, electronic equipment and storage medium

Abstract

The embodiments of the present application disclose a rural environment recognition method, device, electronic device and storage medium. The method includes: acquiring multiple frames of first aerial images and multiple frames of second aerial images; performing target detection on each frame of the first aerial image, respectively, to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image; performing target detection on each frame of the second aerial image, respectively, to obtain at least one second candidate frame and at least one second category in each frame of the first aerial image, wherein at least one second candidate frame corresponds to at least one second category; based on at least one first candidate frame and at least one first category in each frame of the first aerial image, and at least one second candidate frame and at least one second category in each frame of the second aerial image, determine the environmental conditions of the area to be recognized. The embodiments of the present application are conducive to improving the efficiency of rural environment recognition.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and specifically to a rural environment recognition method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the development of science and technology, the economic living standards in cities have been getting higher and higher, the quality of life has been getting better and better, and the environmental governance has been getting better and better. Therefore, people's focus has begun to shift from cities to rural areas.

[0003] In the process of paying attention to rural life, one of the points that people pay more attention to is the quality of rural environment. In order to improve the quality of rural environment, various garbage collections are set up in rural areas, people are regularly sent to clean and maintain them, and rural rivers are regularly combed to manage the environmental sanitation of public areas to improve the quality of rural environment.

[0004] After setting up various environmental protection measures, it is necessary to constantly investigate the rural environmental conditions. At present, the main method of investigating the rural environmental conditions is to send special personnel to check and report the environmental conditions to relevant departments. This rural environmental identification method requires a lot of manpower and material resources and is inefficient. Summary of the invention

[0005] The embodiments of the present application provide a rural environment recognition method, device, electronic device and storage medium, which obtain rural environmental conditions through an image recognition solution and improve the efficiency of rural environment recognition.

[0006] In a first aspect, an embodiment of the present application provides a rural environment identification method, comprising:

[0007] Acquire multiple frames of first aerial images and multiple frames of second aerial images, wherein the multiple frames of first aerial images are obtained by photographing the area to be identified by a drone at a first flight altitude, and the multiple frames of second aerial images are obtained by photographing the area to be identified by the drone at a second flight altitude, the first flight altitude is greater than a first threshold, the second flight altitude is greater than the second threshold and less than the first threshold, and the area to be identified is any rural area in the electronic map;

[0008] Performing target detection on each frame of the first aerial image respectively, to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image, wherein the at least one first candidate frame and the at least one first category correspond one to one;

[0009] Performing target detection on each frame of the second aerial image respectively, to obtain at least one second candidate frame and at least one second category in each frame of the first aerial image, wherein the at least one second candidate frame corresponds to the at least one second category in a one-to-one manner;

[0010] Based on at least one first candidate box and at least one first category in each frame of the first aerial image, and at least one second candidate box and at least one second category in each frame of the second aerial image, the environmental condition of the area to be identified is determined.

[0011] In a second aspect, an embodiment of the present application provides a rural environment identification device, including: a transceiver unit and a processing unit;

[0012] The transceiver unit is used to obtain multiple frames of first aerial images and multiple frames of second aerial images, wherein the multiple frames of first aerial images are obtained by photographing the area to be identified by the drone at a first flight altitude, and the multiple frames of second aerial images are obtained by photographing the area to be identified by the drone at a second flight altitude, the first flight altitude is greater than a first threshold, the second flight altitude is greater than the second threshold and less than the first threshold, and the area to be identified is any rural area in the electronic map;

[0013] The processing unit is configured to perform target detection on each frame of the first aerial image, respectively, to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image, wherein the at least one first candidate frame and the at least one first category correspond to each other in a one-to-one manner;

[0014] Performing target detection on each frame of the second aerial image respectively to obtain a second candidate box and at least one second category in each frame of the first aerial image;

[0015] Based on at least one first candidate box and at least one first category in each frame of the first aerial image, and at least one second candidate box and at least one second category in each frame of the second aerial image, the environmental condition of the area to be identified is determined.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, the processor is connected to a memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer is operable to cause the computer to execute the method described in the first aspect.

[0019] Implementing the embodiments of the present application has the following beneficial effects:

[0020] It can be seen that in the embodiments of the present application, environmental identification of rural areas is performed based on aerial images obtained by drone aerial photography to obtain the environmental conditions of the rural areas. Therefore, through image recognition, there is no need to dispatch professional personnel to conduct rural surveys to obtain the rural environmental conditions, thereby reducing the investment in manpower and material resources and improving the efficiency and accuracy of rural environmental identification. In addition, by identifying environmental conditions separately through aerial images taken at different altitudes, accurate identification of different rural areas can be achieved, further improving the accuracy of rural environmental identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of a rural environment identification process provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of another process of rural environment identification provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of a process for determining a plurality of first shooting positions provided in an embodiment of the present application;

[0025] Figure 4 A block diagram of the functional units of a rural environment identification device provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0030] See also Figure 1 , Figure 1 A flow chart of a rural environment identification method provided in an embodiment of the present application. The method is applied to a rural environment identification device. The method includes the following steps:

[0031] 101: A rural environment recognition device obtains a plurality of frames of first aerial images and a plurality of frames of second aerial images.

[0032] Among them, multiple frames of first aerial images are obtained by photographing the area to be identified by the drone at the first flight altitude, and each frame of the first aerial image is obtained by photographing a partial area in the area to be identified by the drone, and the sum of the partial areas corresponding to the multiple frames of first aerial images is the area to be identified, that is, covering the entire area to be identified.

[0033] Among them, the multiple frames of second aerial images are obtained by photographing the area to be identified by the drone at the second flight altitude, the first flight altitude is greater than the first threshold, the second flight altitude is greater than the second threshold, and less than the first threshold, that is, the first flight altitude is higher than the second flight altitude. Similarly, each frame of the second aerial image is obtained by photographing a partial area of ​​the area to be identified, and the sum of the partial areas corresponding to the multiple frames of the second aerial images is the area to be identified, that is, covering the entire area to be identified.

[0034] The reason for taking images at two different flight altitudes is that the range of the images obtained each time at different altitudes is different, and different altitudes are required for different areas to accurately detect targets. For example, when detecting targets on rivers, high-position shooting is required so that each frame of the image can contain more features related to the river, facilitating accurate target detection of the river; for another example, when identifying garbage collection points, low-position shooting is required so that each frame of the image can contain more features related to the garbage collection points. Therefore, shooting at different flight altitudes allows subsequent target detection of different areas based on different images to improve detection accuracy.

[0035] Optionally, the area to be identified is any rural area in an electronic map. Each rural area may be pre-marked in the electronic map, and any rural area may be obtained by reading the electronic map.

[0036] 102: The rural environment recognition device performs target detection on each frame of the first aerial image to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image.

[0037] Exemplarily, the rural environment recognition device can perform target detection on each frame of the first aerial image by calling a pre-trained neural network. For example, a pre-trained transformer model can be called to perform end-to-end target detection on each frame of the first aerial image to obtain at least one first candidate frame in each frame of the first aerial image and at least one first category corresponding to the at least one candidate frame. The at least one first candidate frame and the at least one first category have a one-to-one correspondence.

[0038] 103: The rural environment recognition device performs target detection on each frame of the second aerial image to obtain at least one second candidate frame and at least one second category in each frame of the first aerial image.

[0039] Similarly, the pre-trained neural network can be used to perform target detection on each frame of the second aerial image to obtain at least one second candidate frame and at least one second category in each frame of the first aerial image. The at least one second candidate frame corresponds to the at least one second category.

[0040] 104: The rural environment recognition device determines the environmental condition of the area to be recognized based on at least one first candidate frame and at least one first category in each frame of the first aerial image, and at least one second candidate frame and at least one second category in each frame of the second aerial image.

[0041] Exemplarily, the first target candidate frame in each frame of the first aerial image is determined, wherein the first target candidate frame is the first candidate frame whose first category is the first target category in the at least one first candidate frame. Optionally, the first target category may be one of: a river, a farmland, a pond, or a lane in front of a village. That is, a first target candidate frame whose first category is the first target category is found from each frame of the first aerial image. It should be noted that the number of first target candidate frames in each frame of the first aerial image is one or more. This application is described by taking one target candidate frame as an example, and taking the first category corresponding to the first target candidate frames in multiple frames of the first aerial image as the same category as an example, that is, assuming that there is only one first target category. Similarly, the subsequent description is also based on the example that the second category corresponding to the second target candidate frames in multiple frames of the second aerial image is the same category, that is, assuming that there is only one second target category.

[0042] Similarly, a second target candidate frame in each frame of the second aerial image is determined, wherein the second category corresponding to the second target candidate frame is the target category, that is, a candidate frame of the second target category is determined from at least one second candidate frame, wherein the second category is one of a garbage collection point or a public activity area.

[0043] Furthermore, the first image selected by the first target candidate box is identified to obtain the environmental conditions corresponding to the first target candidate box, and the second image selected by the second target candidate box is identified to obtain the environmental conditions corresponding to the second target candidate box.

[0044] The following takes a first target candidate frame A as an example to illustrate the process of determining the environmental conditions corresponding to the first target candidate frame A.

[0045] Specifically, the first image corresponding to the first target candidate frame A is identified to obtain the entropy of the first image and the number of foreign objects in the first image, wherein the entropy of the first image reflects the degree of disorder of the objects in the first image.

[0046] Exemplarily, the entropy of the first image is expressed by formula (1):

[0047]

[0048] Among them, H A represents the entropy of the first image corresponding to the first target candidate box A, i represents the pixel value in the first image, ranging from 0 to 255, and p i Represents the number of pixels with pixel value i in the first image.

[0049] From formula (1), we can see that when H A If the value is relatively small, it means that there are more pixels with values ​​close to 255. AWhen the value is relatively large, it means that there are more pixels with pixel values ​​close to 0. When the number of pixels with pixel values ​​close to 255 is relatively large, it means that the distribution of objects in the first image is relatively scattered, and the entire area corresponding to the first image may be distributed with objects, which means that the entire area is relatively messy. When the number of pixels with pixel values ​​close to 0 is relatively large, it means that the distribution of objects in the first image is relatively concentrated, and may be concentrated in one piece, which means that the entire area is relatively neat. Therefore, the entropy of the first image represents the degree of disorder of the objects in the first image, and the larger the entropy, the smaller the disorder, and the smaller the entropy, the greater the disorder.

[0050] Furthermore, the first image selected by the first target candidate frame A may be segmented to obtain the number of foreign objects in the first image, where the foreign objects include plastic bags, pesticide bags, waste, and the like.

[0051] Specifically, the first image is segmented to obtain multiple mask images corresponding to multiple preset foreign objects, wherein each mask image is used to characterize the probability of each pixel in the first image being a preset foreign object corresponding to the mask image; therefore, based on the multiple mask images, the probability that any pixel in the first image belongs to multiple preset foreign objects can be determined; then, the maximum probability of belonging to a certain preset foreign object is taken as the category of the pixel; finally, the pixels of the same category are connected and domains are merged to obtain at least one connected domain, and the number of the at least one connected domain is taken as the number of foreign objects in the first image.

[0052] Finally, the environmental condition corresponding to the first target candidate frame A is determined based on the entropy of the first image and the number of foreign objects in the first image. For example, the number of foreign objects can be used as a benchmark for measuring the environmental condition of the area selected by the first target candidate frame, and the entropy of the first image can be regarded as a proportional coefficient of the benchmark. Therefore, the environmental condition of the first target candidate frame A can be expressed by formula (2):

[0053] E A =s a *sigmoid|H A | Formula (2);

[0054] Among them, E A represents the environmental condition corresponding to the first target candidate box A, s a The number of foreign objects in the first image selected by the first target candidate box A.

[0055] It should be noted that obtaining the environmental condition corresponding to the second target candidate frame in each frame of the second aerial image is similar to obtaining the environmental condition corresponding to the first target candidate frame, and will not be described again. The environmental condition corresponding to the second target candidate frame is determined by obtaining the entropy of the second image corresponding to the second target candidate frame in each frame of the second aerial image and the number of foreign objects in the second image.

[0056] Furthermore, the clarity of each frame of the first aerial image is obtained; the clarity of multiple frames of the first aerial image is normalized to obtain the weight of each frame of the first aerial image, that is, the normalized result of the clarity of each frame of the first aerial image is used as the weight of each frame of the first aerial image. Because clarity plays a vital role in the identification of environmental conditions, it is more in line with actual needs to use the result of clarity normalization as the weight; then, based on the weight of each frame of the first aerial image, the environmental conditions corresponding to the first target candidate frame in multiple frames of the first aerial image are weighted to obtain the first environmental conditions.

[0057] It should be noted that the weighting here refers to weighting the environmental conditions of the first target candidate frames of the same category. For example, a frame of the first aerial image has a first target candidate frame corresponding to a river and a first target candidate frame corresponding to a farmland, and another frame of the first aerial image only has a first target candidate frame corresponding to a river. In this case, the environmental conditions corresponding to the first target candidates corresponding to the river in the two frames of the first aerial images can be weighted to obtain the first environmental condition of the river.

[0058] Similarly, the weight of each frame of the second aerial image is calculated by the clarity of each frame of the second aerial image; then, based on the weight of each frame of the second aerial image, the environmental conditions corresponding to the second target candidate frame in each frame of the second aerial image are weighted to obtain the second environmental conditions. The methods of calculating the weight of the second aerial image and weighting the environmental conditions corresponding to the second target candidate frame are similar to the above-mentioned processing method of the first aerial image, and will not be described again.

[0059] Finally, the first environmental condition and the second environmental condition are summed, and the sum result is used as the environmental condition of the area to be identified.

[0060] In particular, when there are multiple first target categories and multiple second target categories, a first environmental condition will be obtained for each first target category, and similarly, a second environmental condition will be obtained for each second target category. Finally, the first environmental conditions corresponding to all first target categories and the second environmental conditions corresponding to all second target categories are summed up as the environmental condition of the area to be identified.

[0061] It can be seen that in the embodiments of the present application, environmental identification of rural areas is performed based on aerial images obtained by drone aerial photography to obtain the environmental conditions of the rural areas. Therefore, through image recognition, there is no need to dispatch professional personnel to conduct rural surveys to obtain the rural environmental conditions, thereby reducing the investment in manpower and material resources and improving the efficiency and accuracy of rural environmental identification. In addition, by identifying environmental conditions separately through aerial images taken at different altitudes, accurate identification of different rural areas can be achieved, further improving the accuracy of rural environmental identification.

[0062] See also Figure 2 , Figure 2 A flow chart of another rural environment identification method provided in an embodiment of the present application. The method is applied to a rural environment identification device. Figure 1 The same contents as those in the embodiment shown are not described again here. The method of this embodiment includes the following steps:

[0063] 201: The rural environment identification device obtains a layout diagram of the area to be identified.

[0064] Exemplarily, based on the rural archive information, a layout map of the area to be identified can be obtained from the database, that is, the entire distribution of the area to be identified, the altitude of the area to be identified, and the planting conditions of crops in the area to be identified, etc., so as to obtain basic information related to the area to be identified.

[0065] 202: The rural environment identification device determines a first flight altitude and a second flight altitude based on the layout map.

[0066] Exemplarily, the air quality of the area to be identified at the current moment, such as the PM2.5 index, is obtained; based on the air quality, a first candidate flight altitude is determined, wherein the first candidate flight altitude is the highest flight altitude of the drone, that is, the first candidate flight altitude is the highest flight altitude at which the drone can accurately photograph the area to be identified when photographing the area to be identified;

[0067] Exemplarily, based on the layout diagram, the second candidate flight height of the area to be identified is determined. For example, based on the planting conditions of crops, the growth height of crops is determined, and then based on the growth height of the crops, the second candidate flight height is determined, wherein the second candidate flight height is the minimum safe flight height of the drone, that is, the minimum flight height that will not collide with objects in the area to be identified. Therefore, the second candidate flight height is less than the first candidate flight height.

[0068] Further, the first flight altitude and the second flight altitude are selected from between the first candidate flight altitude and the second candidate flight altitude. For example, the first flight altitude and the second flight altitude may be randomly selected.

[0069] 203: The rural environment recognition device determines a plurality of first shooting positions based on the first flight altitude.

[0070] The sum of the shooting areas of the drone at the multiple first shooting positions covers the area to be identified. Exemplarily, the shooting range of the drone at the first flight altitude is obtained; based on the shooting range and the total range of the area to be identified, the multiple first shooting positions are determined so that the drone can completely capture the area to be identified when shooting at the multiple first shooting positions respectively.

[0071] like Figure 3 As shown, based on the first flight altitude and the shooting range of the drone at the first flight altitude, the first shooting positions P1, P2, P3 and P4 can be determined. It can be seen that when the drone shoots at the first shooting positions P1, P2, P3 and P4 respectively, the entire area to be identified can be photographed.

[0072] 204: The rural environment recognition device determines a plurality of second shooting positions based on the second flight altitude.

[0073] The sum of the shooting areas of the drone at the plurality of second shooting positions covers the area to be identified. Similarly, similar to determining the plurality of first shooting positions, the plurality of second shooting positions can be determined based on the shooting range of the drone at the second flight altitude, which will not be described again.

[0074] 205: The rural environment identification device sends a first shooting instruction and a second shooting instruction to the drone.

[0075] Among them, the first shooting instruction is used to instruct the drone to shoot at the multiple first shooting positions to obtain the multiple frames of first aerial images, and the second shooting instruction is used to instruct the drone to shoot at the multiple second shooting positions to obtain the multiple frames of second aerial images.

[0076] 206: The rural environment identification device obtains a plurality of frames of first aerial images and a plurality of frames of second aerial images from the UAV.

[0077] Among them, the multiple frames of first aerial images are obtained by photographing the area to be identified by a drone at a first flight altitude, and the multiple frames of second aerial images are obtained by photographing the area to be identified by the drone at a second flight altitude. The first flight altitude is greater than a first threshold, the second flight altitude is greater than the second threshold and less than the first threshold, and the area to be identified is any rural area in the electronic map.

[0078] 207: The rural environment recognition device performs target detection on each frame of the first aerial image to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image, wherein the at least one first candidate frame and the at least one first category have a one-to-one correspondence

[0079] 208: The rural environment recognition device performs target detection on each frame of the second aerial image to obtain at least one second candidate frame and at least one second category in each frame of the first aerial image, wherein the at least one second candidate frame corresponds to the at least one second category in a one-to-one manner.

[0080] 209: The rural environment recognition device determines the environmental conditions of the area to be recognized based on at least one first candidate frame and at least one first category in each frame of the first aerial image, and at least one second candidate frame and at least one second category in each frame of the second aerial image.

[0081] It can be seen that in the embodiments of the present application, environmental identification of rural areas is performed based on aerial images obtained by drone aerial photography to obtain the environmental conditions of the rural areas. Therefore, through image recognition, there is no need to dispatch professional personnel to conduct rural surveys to obtain the rural environmental conditions, thereby reducing the investment in manpower and material resources and improving the efficiency and accuracy of rural environmental identification. In addition, by identifying environmental conditions separately through aerial images taken at different altitudes, accurate identification of different rural areas can be achieved, further improving the accuracy of rural environmental identification.

[0082] See also Figure 4 , Figure 4 The functional unit composition block diagram of a rural environment identification device provided in an embodiment of the present application. The rural environment identification device 400 includes: a transceiver unit 401 and a processing unit 402;

[0083] The transceiver unit 401 is used to obtain multiple frames of first aerial images and multiple frames of second aerial images, wherein the multiple frames of first aerial images are obtained by photographing the area to be identified by a drone at a first flight altitude, and the multiple frames of second aerial images are obtained by photographing the area to be identified by the drone at a second flight altitude, the first flight altitude is greater than a first threshold, the second flight altitude is greater than the second threshold and less than the first threshold, and the area to be identified is any rural area in the electronic map;

[0084] Processing unit 402, for a first threshold, the area to be identified is any rural area in the electronic map;

[0085] Performing target detection on each frame of the first aerial image respectively, to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image, wherein the at least one first candidate frame and the at least one first category correspond one to one;

[0086] Performing target detection on each frame of the second aerial image respectively, to obtain at least one second candidate frame and at least one second category in each frame of the first aerial image, wherein the at least one second candidate frame corresponds to the at least one second category in a one-to-one manner;

[0087] Based on at least one first candidate box and at least one first category in each frame of the first aerial image, and at least one second candidate box and at least one second category in each frame of the second aerial image, the environmental condition of the area to be identified is determined.

[0088] In some possible implementations, before the transceiver unit 401 acquires the multiple frames of the first aerial image and the multiple frames of the second aerial image, the processing unit 402 is further configured to:

[0089] Obtaining a layout diagram of the area to be identified;

[0090] Based on the layout diagram, determining a first flight altitude and a second flight altitude;

[0091] Based on the first flight altitude, determining a plurality of first shooting positions, wherein the sum of the shooting areas of the UAV at the plurality of first shooting positions covers the area to be identified;

[0092] Based on the second flight altitude, determining a plurality of second shooting positions, wherein the sum of the shooting areas of the UAV at the plurality of second shooting positions covers the area to be identified;

[0093] A first shooting instruction and a second shooting instruction are sent to the drone, wherein the first shooting instruction is used to instruct the drone to shoot at the multiple first shooting positions to obtain the multiple frames of first aerial images, and the second shooting instruction is used to instruct the drone to shoot at the multiple second shooting positions to obtain the multiple frames of second aerial images.

[0094] In some possible implementations, in determining the first flying height and the second flying height based on the layout diagram, the processing unit 402 is specifically configured to:

[0095] Acquire the air quality of the area to be identified;

[0096] Based on the air quality, determining a first candidate flight altitude of the area to be identified;

[0097] Based on the layout diagram, determining a second candidate flight height of the to-be-identified area, wherein the second candidate flight height is smaller than the first candidate flight height;

[0098] The first flight altitude and the second flight altitude are selected from among the first candidate flight altitude and the second candidate flight altitude.

[0099] In some possible implementations, in determining the environmental condition of the to-be-identified area based on at least one first candidate box and at least one first category in each frame of the first aerial image, and at least one second candidate box and at least one second category in each frame of the second aerial image, the processing unit 402 is specifically configured to:

[0100] Determine a first target candidate frame in each frame of the first aerial image, wherein the first category corresponding to the first target candidate frame is a first target category;

[0101] Determine a second target candidate frame in each frame of the second aerial image, wherein the second category corresponding to the second target candidate frame is the second target category;

[0102] Recognize the first image selected by the first target candidate frame to obtain an environmental condition corresponding to the first target candidate frame;

[0103] Weighting the environmental condition corresponding to the first target candidate frame in each frame of the first aerial image to obtain a first environmental condition;

[0104] Recognize the second image selected by the second target candidate frame to obtain an environmental condition corresponding to the first target candidate frame;

[0105] Weighting the environmental condition corresponding to the second target candidate box in each frame of the second aerial image to obtain a second environmental condition;

[0106] The first environmental condition and the second environmental condition are summed to obtain the environmental condition of the area to be identified.

[0107] In some possible implementations, in terms of identifying the first image selected by the first target candidate frame to obtain an environmental condition corresponding to the first target candidate frame, the processing unit 402 is specifically configured to:

[0108] Identify the first image selected by the first target candidate box in each frame of the first aerial image to obtain the entropy of the first image and the number of foreign objects in the first image, wherein the entropy of the first image is used to characterize the degree of disorder of the object in the first image;

[0109] An environmental condition corresponding to the first target candidate box is determined according to the entropy of the first image and the number of foreign objects in the first image.

[0110] In some possible implementations, in terms of weighting the environmental condition corresponding to the first target candidate box in each frame of the first aerial image to obtain the first environmental condition, the processing unit 402 is specifically configured to:

[0111] Obtaining the clarity of each frame of the first aerial image;

[0112] Normalizing the clarity of the multiple frames of the first aerial image to obtain a weight of each frame of the first aerial image;

[0113] According to the weight of each frame of the first aerial image, the environmental condition corresponding to the first target candidate frame in each frame of the first aerial image is weighted to obtain the first environmental condition.

[0114] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes a transceiver 501, a processor 502 and a memory 503. They are connected via a bus 504. The memory 503 is used to store computer programs and data, and can transmit the data stored in the memory 503 to the processor 502.

[0115] The processor 502 is used to read the computer program in the memory 503 and perform the following operations:

[0116] In some possible implementations,

[0117] Specifically, the transceiver 501 may be Figure 4 The transceiver unit 401 of the rural environment identification device 400 of the embodiment described above, the processor 502 may be Figure 4 The processing unit 402 of the rural environment identification device 400 of the described embodiment.

[0118] It should be understood that the electronic devices in this application may include smart phones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, PDAs, laptops, mobile Internet devices MID (Mobile Internet Devices, referred to as: MID) or wearable devices, etc. The above electronic devices are only examples, not exhaustive, and include but are not limited to the above electronic devices. In practical applications, the above electronic devices may also include: smart vehicle terminals, computer equipment, etc.

[0119] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement part or all of the steps of any rural environment identification method recorded in the above method embodiments.

[0120] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any rural environment identification method recorded in the above method embodiments.

[0121] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0122] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0123] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.

[0126] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0127] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0128] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A rural environment identification method, characterized in that: include: Acquire multiple frames of first aerial images and multiple frames of second aerial images, wherein the multiple frames of first aerial images are obtained by photographing the area to be identified by a drone at a first flight altitude, and the multiple frames of second aerial images are obtained by photographing the area to be identified by the drone at a second flight altitude, the first flight altitude is greater than a first threshold, the second flight altitude is greater than the second threshold and less than the first threshold, and the area to be identified is any rural area in the electronic map; Performing target detection on each frame of the first aerial image respectively, to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image, wherein the at least one first candidate frame and the at least one first category correspond one to one; Performing target detection on each frame of the second aerial image respectively, to obtain at least one second candidate frame and at least one second category in each frame of the second aerial image, wherein the at least one second candidate frame corresponds to the at least one second category in a one-to-one manner; Based on at least one first candidate box and at least one first category in each frame of the first aerial image, and at least one second candidate box and at least one second category in each frame of the second aerial image, the environmental condition of the area to be identified is determined.

2. The method according to claim 1, characterized in that Before acquiring the multiple frames of the first aerial image and the multiple frames of the second aerial image, the method further includes: Obtaining a layout diagram of the area to be identified; Based on the layout diagram, determining a first flight altitude and a second flight altitude; Based on the first flight altitude, determining a plurality of first shooting positions, wherein the sum of the shooting areas of the UAV at the plurality of first shooting positions covers the area to be identified; Based on the second flight altitude, determining a plurality of second shooting positions, wherein the sum of the shooting areas of the UAV at the plurality of second shooting positions covers the area to be identified; A first shooting instruction and a second shooting instruction are sent to the drone, wherein the first shooting instruction is used to instruct the drone to shoot at the multiple first shooting positions to obtain the multiple frames of first aerial images, and the second shooting instruction is used to instruct the drone to shoot at the multiple second shooting positions to obtain the multiple frames of second aerial images.

3. The method according to claim 2, characterized in that Determining a first flying height and a second flying height based on the layout diagram includes: Acquire the air quality of the area to be identified; Based on the air quality, determining a first candidate flight altitude of the area to be identified; Based on the layout diagram, determining a second candidate flight height of the to-be-identified area, wherein the second candidate flight height is smaller than the first candidate flight height; The first flight altitude and the second flight altitude are selected from among the first candidate flight altitude and the second candidate flight altitude.

4. The method according to any one of claims 1 to 3, characterized in that The determining the environmental condition of the to-be-identified area based on at least one first candidate frame and at least one first category in each frame of the first aerial image, and at least one second candidate frame and at least one second category in each frame of the second aerial image, comprises: Determine a first target candidate frame in each frame of the first aerial image, wherein the first category corresponding to the first target candidate frame is a first target category; Determine a second target candidate frame in each frame of the second aerial image, wherein the second category corresponding to the second target candidate frame is the second target category; Recognize the first image selected by the first target candidate frame to obtain an environmental condition corresponding to the first target candidate frame; Weighting the environmental condition corresponding to the first target candidate frame in each frame of the first aerial image to obtain a first environmental condition; Recognize the second image selected by the second target candidate frame to obtain an environmental condition corresponding to the first target candidate frame; Weighting the environmental condition corresponding to the second target candidate box in each frame of the second aerial image to obtain a second environmental condition; The first environmental condition and the second environmental condition are summed to obtain the environmental condition of the area to be identified.

5. The method according to claim 4, characterized in that The step of recognizing the first image selected by the first target candidate frame to obtain an environmental condition corresponding to the first target candidate frame includes: Identify the first image selected by the first target candidate box in each frame of the first aerial image to obtain the entropy of the first image and the number of foreign objects in the first image, wherein the entropy of the first image is used to characterize the degree of disorder of the object in the first image; An environmental condition corresponding to the first target candidate box is determined according to the entropy of the first image and the number of foreign objects in the first image.

6. The method according to claim 4, characterized in that The step of weighting the environmental condition corresponding to the first target candidate frame in each frame of the first aerial image to obtain the first environmental condition includes: Obtaining the clarity of each frame of the first aerial image; Normalizing the clarity of the multiple frames of the first aerial image to obtain a weight of each frame of the first aerial image; According to the weight of each frame of the first aerial image, the environmental condition corresponding to the first target candidate frame in each frame of the first aerial image is weighted to obtain the first environmental condition.

7. A rural environment identification device, characterized in that: include: Transceiver unit and processing unit; The transceiver unit is used to obtain multiple frames of first aerial images and multiple frames of second aerial images, wherein the multiple frames of first aerial images are obtained by photographing the area to be identified by the drone at a first flight altitude, and the multiple frames of second aerial images are obtained by photographing the area to be identified by the drone at a second flight altitude, the first flight altitude is greater than a first threshold, the second flight altitude is greater than the second threshold and less than the first threshold, and the area to be identified is any rural area in the electronic map; The processing unit is configured to perform target detection on each frame of the first aerial image, respectively, to obtain at least one first candidate frame and at least one first category in each frame of the first aerial image, wherein the at least one first candidate frame and the at least one first category correspond to each other in a one-to-one manner; Performing target detection on each frame of the second aerial image respectively to obtain a second candidate box and at least one second category in each frame of the second aerial image; Based on at least one first candidate box and at least one first category in each frame of the first aerial image, and at least one second candidate box and at least one second category in each frame of the second aerial image, the environmental condition of the area to be identified is determined.

8. The device according to claim 7, characterized in that Before the transceiver unit acquires the multiple frames of the first aerial image and the multiple frames of the second aerial image, the processing unit is further used to: Obtaining a layout diagram of the area to be identified; Based on the layout diagram, determining a first flight altitude and a second flight altitude; Based on the first flight altitude, determining a plurality of first shooting positions, wherein the sum of the shooting areas of the UAV at the plurality of first shooting positions covers the area to be identified; Based on the second flight altitude, determining a plurality of second shooting positions, wherein the sum of the shooting areas of the UAV at the plurality of second shooting positions covers the area to be identified; A first shooting instruction and a second shooting instruction are sent to the drone, wherein the first shooting instruction is used to instruct the drone to shoot at the multiple first shooting positions to obtain the multiple frames of first aerial images, and the second shooting instruction is used to instruct the drone to shoot at the multiple second shooting positions to obtain the multiple frames of second aerial images.

9. An electronic device, characterized in that: include: A processor and a memory, the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 6.

Images