Unmanned aerial vehicle-based sea floating garbage identification method and system, medium and program product

Abstract

The invention provides a sea floating garbage identification method and system based on an unmanned aerial vehicle, a medium and a program product, and relates to the technical field of image identification processing. The method comprises the steps that an unmanned aerial vehicle obtains multiple pieces of sea surface image information, then the image information is input into a sea floating garbage recognition model, the model is constructed by conducting deep learning on a large number of historical sea surface image information sets of which the sea floating garbage information is manually marked, information such as the number and types of the sea floating garbage can be accurately determined, and the sea floating garbage recognition efficiency is improved. And a visual sea floating garbage distribution map is generated in combination with sea floating garbage position information obtained by the positioning module and the identified sea floating garbage information. And finally, the unmanned aerial vehicle is remotely controlled to adjust flight parameters according to the sea floating garbage distribution diagram, the sea floating garbage is fished, and the target of accurately recognizing the sea floating garbage under the complex weather condition can be achieved through the method.

Description

Technical Field

[0001] This application relates to the technical field of image recognition processing, and in particular, to a method, system, medium, and program product for identifying marine floating garbage based on an unmanned aerial vehicle (UAV). Background Art

[0002] Effective monitoring and management of marine floating waste have become a key link in protecting the marine environment.

[0003] Existing monitoring methods mainly rely on satellite remote sensing. Satellite remote sensing for monitoring marine floating waste is mainly achieved through satellites equipped with various sensors. An optical sensor is one of the commonly used ones. During the operation of the optical camera on the satellite, the ocean surface is photographed using the visible light band. When the weather is clear and the lighting conditions are good, the optical sensor can capture the image information of the ocean surface. Staff will use professional image processing software to preprocess the image information, and through image analysis technology, identify the possible marine floating waste in the image.

[0004] However, the existing technology has a high cost and is often easily affected by cloudy and foggy weather, resulting in low recognition accuracy. Summary of the Invention

[0005] This application provides a method, system, medium, and program product for identifying marine floating garbage based on an unmanned aerial vehicle, which is used to achieve the goal of accurately identifying marine floating garbage under complex weather conditions.

[0006] In a first aspect, this application provides a method for identifying marine floating garbage based on an unmanned aerial vehicle, which is applied to a marine floating garbage identification system. The method includes: obtaining a plurality of sea surface image information through the unmanned aerial vehicle; inputting the sea surface image information into a marine floating garbage identification model to determine marine floating garbage information, where the marine floating garbage identification model is previously constructed through deep learning from a plurality of historical sea surface image information sets with manually labeled marine floating garbage information, and the marine floating garbage information at least includes the quantity and type of marine floating garbage; generating a marine floating garbage distribution map by combining the marine floating garbage position information and the marine floating garbage information, where the marine floating garbage position information is obtained according to a positioning module; remotely controlling the flight parameters of the unmanned aerial vehicle to perform garbage collection according to the marine floating garbage distribution map.

[0007] By adopting the above technical solution, the unmanned aerial vehicle (UAV) can obtain multiple sea surface image information, providing a data basis for the identification of marine floating garbage. The image information is input into the marine floating garbage identification model constructed through deep learning, and its identification ability trained based on a large number of manually annotated historical image information sets is used to accurately determine the quantity and type of marine floating garbage. Combining with the position information obtained by the positioning module, a distribution map of marine floating garbage is generated, visually presenting the distribution of marine floating garbage. The UAV is remotely controlled to salvage according to the map, realizing an integrated process from monitoring to salvage. Compared with the high cost and resolution limitations of satellite remote sensing, as well as the low efficiency of manual inspection, it can also avoid the interference of clouds and fog on garbage identification in the high-altitude environment. This method has lower costs, higher resolution, and can accurately locate and salvage, improving the efficiency and accuracy of marine floating garbage monitoring and salvage.

[0008] Combined with some embodiments of the first aspect, in some embodiments, before the step of obtaining multiple sea surface image information by the UAV, it further includes: obtaining light intensity information through multiple sensors; combining the light intensity information, adjusting the camera parameters of the UAV according to a preset parameter adjustment strategy library; if the light intensity information is greater than a preset strong light intensity threshold, reducing the aperture size and decreasing the sensitivity of the UAV camera according to a preset strong light amplitude; if the light intensity information is less than a preset weak light intensity threshold, increasing the aperture of the UAV camera and increasing the sensitivity according to a preset weak light amplitude.

[0009] By adopting the above technical solution, when the light intensity is greater than the preset strong light intensity threshold, the aperture is reduced and the sensitivity is decreased to avoid overexposure of the image caused by overly bright light and ensure that the details of the objects in the image are clearly distinguishable; when the light intensity is less than the preset weak light intensity threshold, the aperture is increased and the sensitivity is increased to ensure that an image with sufficient brightness can be obtained even in an environment with insufficient light. In this way, regardless of how the light conditions change, it can ensure that the sea surface images obtained by the UAV are clear and of high quality, providing high-quality image data for the subsequent accurate identification of marine floating garbage by the marine floating garbage identification model and improving the identification accuracy.

[0010] Combined with some embodiments of the first aspect, in some embodiments, after the step of increasing the aperture of the UAV camera and increasing the sensitivity according to a preset weak light amplitude if the light intensity information is less than the preset weak light intensity threshold, it further includes: combining the multiple sea surface image information and a preset meteorological identification model to determine real-time sea area environmental condition information; combining the preset sea area safety level to rate the sea area environmental condition information to determine the current sea area safety level; if the current sea area safety level is greater than a preset safety threshold, sending an evacuation instruction to the UAV.

[0011] By adopting the above technical solution, this process ensures the flight safety of the UAV, avoids the UAV from performing tasks in dangerous sea area environments such as storms and strong convections, reduces the risk of UAV damage, and extends the service life of the UAV.

[0012] Combined with some embodiments of the first aspect, in some embodiments, after the step of generating a sea drift garbage distribution map by combining the sea drift garbage position information and the sea drift garbage information, the method further includes: obtaining the summary information of the sea drift garbage quantity and the sea drift garbage type information of a plurality of target areas; if the summary information of the sea drift garbage quantity exceeds a preset quantity threshold or the sea drift garbage type information belongs to a preset danger level, sending a warning message to a preset management terminal.

[0013] By adopting the above technical solution, once the summary information of the sea drift garbage quantity exceeds the preset quantity threshold, it means that the sea drift garbage pollution in this area is serious; if the sea drift garbage type information belongs to the preset danger level, it indicates that the garbage may pose a greater potential hazard to the marine ecosystem or human activities. At this time, sending a warning message to the preset management terminal can enable the management personnel to timely grasp the serious pollution or dangerous situation of the sea drift garbage, so as to quickly formulate and implement targeted cleaning and treatment measures, and effectively reduce the negative impact of the sea drift garbage on the marine environment, marine organisms and coastal economic activities.

[0014] Combined with some embodiments of the first aspect, in some embodiments, after the step of generating a sea drift garbage distribution map by combining the sea drift garbage position information and the sea drift garbage information, the method further includes: obtaining a plurality of water flow information and wind direction information through sensors, where the water flow information includes water flow speed and direction data, and the wind direction information includes wind direction and wind speed data; according to the water flow information, the wind direction information and the sea drift garbage type, combining a water flow model and a wind direction model to determine the drift path information of the sea drift garbage.

[0015] By adopting the above technical solution, the sensor obtains information such as water flow speed, direction, wind direction and wind speed, combines the sea drift garbage type, and with the help of the water flow model and the wind direction model, can accurately determine the drift path information of the sea drift garbage. Water flow and wind direction are the key factors affecting the movement of sea drift garbage. Different types of sea drift garbage have different movement laws under the action of water flow and wind force due to differences in their materials, shapes, etc. Through model calculation, the drift trajectory of the sea drift garbage can be simulated. This not only helps to predict the whereabouts of the sea drift garbage in advance, provides forward-looking information for marine environmental protection and management, but also enables relevant departments to plan the cleaning work in advance and improve the cleaning efficiency.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of determining the drift path information of the floating garbage at sea based on the water flow information, the wind direction information and the type of floating garbage at sea, combined with the water flow model and the wind direction model, the step also includes: obtaining the flight time information of the drone flying to the location of the floating garbage at sea; determining the position change of the floating garbage at sea within the flight time information based on the drift path information; determining the final position of the floating garbage at sea when the drone arrives at the location of the floating garbage based on the position change; and controlling the drone to fly directly to the final position for garbage salvage.

[0017] By adopting the above technical solution, the drone flies directly to the final location for salvage, avoiding the situation where the drone cannot salvage the floating garbage after reaching the initial location due to the movement of floating garbage with the water flow and wind direction. This greatly improves the success rate of drone salvage, reduces the ineffective flight time of drones, reduces energy consumption, and improves the efficiency of floating garbage salvage, so that limited resources can be used more efficiently, and floating garbage can be cleaned more effectively to protect the marine environment.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of acquiring multiple sea surface image information through a drone, it also includes: using an image preprocessing module to perform preprocessing operations on the sea surface image information, and the preprocessing operations include grayscale processing, noise reduction processing and size normalization processing.

[0019] By adopting the above technical solution, the pre-processed images can enable the marine debris identification model to identify marine debris more accurately and quickly, improve the accuracy and efficiency of marine debris information determination, and provide reliable data support for subsequent generation of distribution maps, salvage and other links.

[0020] In a second aspect, the present application provides a system for identifying marine debris, the system comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, the one or more processors call the computer instructions to enable the system to perform the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a marine debris identification system, causes the marine debris identification system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computer program product. When the computer program product is run on a marine debris identification system, the marine debris identification system executes the method described in the first aspect and any possible implementation of the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Since the technical means of using drones to obtain sea surface images, a sea drift garbage recognition model constructed by deep learning to recognize sea drift garbage, generating a distribution map in combination with positioning information, and controlling the drone to salvage according to the map are adopted, the technical problems of high cost, limited resolution of existing satellite remote sensing, and low efficiency of manual inspection that are difficult to cover vast sea areas are effectively solved. Furthermore, the technical effect of monitoring and salvaging sea drift garbage at low cost, high resolution, efficiently and accurately is achieved, and the treatment efficiency of sea drift garbage is significantly improved.

[0024] 2. Since the technical means of determining the sea area environmental conditions by combining sea surface images and a meteorological recognition model, rating according to a preset safety level, and sending an evacuation instruction in case of danger are adopted, the technical problems that drones face safety risks in complex and changeable sea area environments, which may lead to equipment damage and threats to personnel safety, are effectively solved. Furthermore, the technical effect of ensuring the flight safety of drones, reducing equipment loss, protecting the safety of operators, and ensuring the safe and orderly development of sea drift garbage monitoring work is achieved.

[0025] 3. Since the technical means of using sensors to obtain water flow and wind direction information, combining the types of sea drift garbage, and determining the drift path with the help of water flow and wind direction models are adopted, the technical problems that the moving direction of sea drift garbage cannot be predicted in advance, resulting in lack of foresight and low efficiency in the cleaning work, are effectively solved. Furthermore, the technical effect of being able to predict the whereabouts of sea drift garbage in advance, providing a planning basis for marine environmental protection and management, improving the cleaning efficiency, and reducing the harm of sea drift garbage is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of a method for identifying sea drift garbage based on drones in the embodiments of the present application; Figure 2 is another flowchart of a method for identifying sea drift garbage based on drones in the embodiments of the present application; Figure 3 is a schematic structural diagram of an entity device of a sea drift garbage recognition system in the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The terms used in the following embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. As used in the specification and appended claims of this application, the singular forms "a", "an", "the", "above-mentioned", "said", "this" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0028] Hereinafter, the terms "first" and "second" are only for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of this application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0029] For ease of understanding, the method provided in this embodiment is described in a process below. Please refer to Figure 1 , which is a schematic flowchart of a method for identifying marine floating garbage based on an unmanned aerial vehicle in an embodiment of this application.

[0030] S101. Obtain a plurality of sea surface image information through an unmanned aerial vehicle; The marine floating garbage identification system first starts the unmanned aerial vehicle. The unmanned aerial vehicle is equipped with an image acquisition system, which can be used to obtain sea surface image information. The unmanned aerial vehicle is equipped with a multi-camera combination, including a high-resolution optical camera, a thermal imaging camera, and a multi-spectral camera. The high-resolution optical camera is used to photograph the sea surface under normal lighting conditions. It has high pixels, a large aperture, and good optical anti-shake function, and can capture tiny garbage objects on the sea surface. The multi-spectral camera can obtain image information in different spectral bands. Garbage of different materials will exhibit unique reflection and absorption characteristics in different spectra. The multi-spectral camera can use these characteristics to classify and identify the garbage.

[0031] The system will pre-plan the flight path of the unmanned aerial vehicle according to the scope, shape of the target sea area and past garbage distribution data. At the same time, the unmanned aerial vehicle is also equipped with a variety of sensors for monitoring environmental parameters. For example, the wind speed sensor can monitor the wind speed and direction at the position where the unmanned aerial vehicle is located in real time, the pressure sensor can measure the flight altitude and pressure change of the unmanned aerial vehicle, and the temperature and humidity sensor can monitor the temperature and humidity of the environment. These sensor data will be transmitted back to the marine floating garbage identification system in real time, and the system will adjust the flight parameters and shooting parameters of the unmanned aerial vehicle according to these data. To cope with the complex marine environment, the unmanned aerial vehicle has the functions of waterproof and anti-salt fog. Its shell is made of special waterproof materials, which can effectively prevent the erosion of sea water.

[0032] In some embodiments, before this step, the system obtains real-time illumination intensity information with the aid of multiple sensors. These sensors are placed at appropriate positions on the unmanned aerial vehicle (UAV) and can comprehensively and accurately sense the illumination changes in the surrounding environment. After receiving the illumination intensity information, the system immediately makes intelligent adjustments to the camera parameters of the UAV by combining a preset parameter adjustment strategy library, which is constructed based on a large amount of experimental data and practical application experience and formulates detailed and accurate parameter adjustment schemes for different illumination intensity ranges. When the illumination intensity information is greater than the preset strong light intensity threshold, it indicates that the current environment is in a state of strong light irradiation. In this case, to avoid overexposure of the captured image caused by too bright light, which affects the recognition effect of marine floating garbage, the system automatically reduces the aperture size of the UAV camera according to the preset strong light amplitude. A smaller aperture can reduce the amount of light entering the camera, thereby preventing the image from losing details due to excessive light. The system also reduces the sensitivity of the camera. After the sensitivity is reduced, the sensitivity of the camera to light decreases, further avoiding the occurrence of overexposure and ensuring that the details of the objects in the image are clearly distinguishable, providing high-quality image data for the subsequent marine floating garbage recognition model. If the illumination intensity information is less than the preset weak light intensity threshold, it means that the light is insufficient. To obtain an image with sufficient brightness in this low-light environment, the system increases the aperture of the UAV camera according to the preset weak light amplitude. A larger aperture allows more light to enter the camera, increasing the overall brightness of the image. The system increases the sensitivity of the camera, making the camera more sensitive to weak light, thereby compensating for the lack of light and ensuring that the captured sea surface image has sufficient clarity and brightness, providing effective data support for the recognition of marine floating garbage.

[0033] After completing the adjustment of the camera parameters, the system determines the real-time sea area environmental condition information by combining multiple acquired sea surface image information and a preset meteorological recognition model. The meteorological recognition model is trained through deep learning on a large amount of historical meteorological data, sea surface image features, and the corresponding environmental conditions. It can extract key features such as cloud thickness, wave form, and sea fog concentration from the sea surface image and combine meteorological data in the same time period, such as temperature, pressure, and humidity, to accurately infer the real-time environmental conditions of the current sea area, including detailed information such as weather type (sunny, cloudy, rainy, foggy, etc.), wind force level, and wave height.

[0034] After determining the real-time sea area environmental condition information, the system will rate it in combination with the preset sea area safety level to determine the safety level of the current sea area. The preset sea area safety level is a comprehensive and scientific evaluation standard formulated based on various factors of the marine environment, such as weather conditions, sea conditions, and marine biological activities. This standard divides the sea area safety level into different levels, and each level corresponds to a specific environmental risk degree. If the current sea area safety level is greater than the preset safety threshold, it indicates that the sea area environment is in a dangerous state. For example, extreme situations such as storms and severe convective weather may occur. In this case, to ensure the safety of the UAV, avoid equipment damage and data loss, the system will immediately send an evacuation instruction to the UAV. After receiving the instruction, the UAV will quickly adjust its flight path according to the preset evacuation procedure and safely evacuate the dangerous sea area. This process not only protects the UAV equipment but also ensures the safety of the operator, guaranteeing the safe and orderly conduct of the sea drift garbage monitoring work.

[0035] S102. Input the sea surface image information into the sea drift garbage recognition model to determine the sea drift garbage information, where the sea drift garbage recognition model is previously constructed through deep learning from multiple historical sea surface image information sets with manually annotated sea drift garbage information, and the sea drift garbage information at least includes the quantity and type of sea drift garbage. After receiving the sea surface image information collected by the UAV, the sea drift garbage recognition system will input these image data into the pre-constructed sea drift garbage recognition model. The construction of this model is based on deep learning algorithms with a convolutional neural network (CNN) as the core architecture. In the training stage, the system uses a large number of historical sea surface image information sets with manually and finely annotated sea drift garbage information. The annotation content covers the accurate position, shape outline, quantity statistics of the garbage, and clear type annotations, such as common sea drift garbage categories like plastic, metal, wood, fabric, etc.

[0036] Before the image is input into the model, the system first preprocesses the sea surface image. Since the images collected by the UAV may have quality problems due to factors such as lighting conditions, shooting angles, and marine environment interference, the preprocessing link is crucial. The system first performs grayscale processing, converting the color image into a grayscale image, reducing the data dimension while retaining the key information of the image, which helps the model extract features faster. Then, the Gaussian filtering algorithm is used to remove the Gaussian noise in the image. By calculating the weighted average of the pixel neighborhood, the image is effectively smoothed while trying to retain the image edges.

[0037] The pre - processed image enters the sea - floating garbage recognition model. The convolutional layer in the model performs convolution operations by sliding the convolution kernel on the image, automatically extracting the features in the image. Convolution kernels of different sizes and parameters are responsible for capturing features at different levels. After alternating stacking of multiple convolutional layers and pooling layers, the features of the image are deeply extracted and compressed. Subsequently, the fully - connected layer comprehensively analyzes these features and outputs the quantity and type information of sea - floating garbage. To improve the accuracy and generalization ability of the model, the system adopts a variety of techniques during the training process.

[0038] S103. Generate a sea - floating garbage distribution map by combining the sea - floating garbage location information and the sea - floating garbage information, where the sea - floating garbage location information is obtained according to the positioning module. After the sea - floating garbage recognition system obtains the quantity, type information of sea - floating garbage and the location information provided by the positioning module, it begins to generate an intuitive and accurate sea - floating garbage distribution map. The data fusion module in the system is responsible for integrating these two types of key information. The positioning module usually adopts the Global Navigation Satellite System (GNSS), such as GPS, Beidou, etc. By receiving satellite signals, it can accurately obtain the geographical location coordinates of the UAV when shooting sea surface images in real time, with an accuracy of up to sub - meter level or even higher. While the UAV is collecting images, the positioning module records the corresponding longitude and latitude information and transmits it back to the sea - floating garbage recognition system together with the image data. The system associates the recognition results such as the quantity and type of sea - floating garbage with the corresponding position coordinates. To more clearly display the distribution of sea - floating garbage, the system adopts Geographic Information System (GIS) technology. GIS software has powerful spatial data processing and visualization capabilities, and can map the location information of sea - floating garbage onto an electronic nautical chart or a satellite image base map. When drawing the distribution map, the system sets different visualization identifiers according to the type and quantity of sea - floating garbage. For areas with a large amount of plastic garbage, larger - sized circular icons may be used, with a color of blue; for metal garbage areas, square icons are used, with a color set to gray. Different icon shapes and colors are used to distinguish different types of sea - floating garbage, making the distribution map more intuitive and easy to understand.

[0039] The system can also perform statistical analysis on the distribution data of sea - floating garbage. For example, calculate the density of sea - floating garbage in different regions and display the density difference on the distribution map by means of color gradient. The darker the color, the higher the density of sea - floating garbage in that area, and vice versa. In addition, the system can generate a dynamic distribution map to show the change of sea - floating garbage distribution according to the time series. This helps relevant departments analyze the movement trend and diffusion law of sea - floating garbage and provides a basis for formulating more effective cleaning and governance strategies.

[0040] In some embodiments, after the generation of the sea floating garbage distribution map is completed, the sea floating garbage identification system can carry out information collection and statistics for multiple target areas. These target areas are demarcated based on the sensitivity of the marine ecological environment, the frequency of human activities, and the distribution patterns of past sea floating garbage. The system summarizes the quantity and accurately identifies the type information of sea floating garbage distributed in different target areas. With the help of the sea floating garbage identification model constructed in the early stage, the model can accurately distinguish different types of sea floating garbage such as plastic, metal, glass, fabric, etc., and accurately count the number of garbage in each area. In order to be able to timely discover the serious impact of sea floating garbage on the marine environment, the system pre-sets the quantity threshold and danger level standard. The quantity threshold is determined by analyzing the carrying capacity of the marine ecosystem, data of past sea floating garbage pollution incidents, and potential impact assessments on marine life and human activities. When the summary information of the number of sea floating garbage in a target area exceeds this threshold, it means that the sea floating garbage pollution in the area has reached a relatively serious level and may cause significant damage to the marine ecosystem. The danger level standard is set according to the type of sea floating garbage. For example, electronic waste containing heavy metals, highly corrosive chemical waste, etc., once these wastes enter the ocean, will pose a major threat to the survival of marine life, marine water quality and coastal human activities, and are therefore classified as a preset danger level. Once the summary information on the amount of floating garbage at sea exceeds the preset quantity threshold, or the type of floating garbage at sea belongs to the preset danger level, the floating garbage identification system will immediately activate the early warning mechanism. The system will send early warning information to the preset management end, which can be the command center of the marine environmental protection department, the data monitoring platform of relevant scientific research institutions, or the operation management unit responsible for marine cleaning. The early warning information contains detailed target area location information, the amount and type of floating garbage at sea, and the current pollution status assessment. This information is transmitted to the management end through a reliable communication network. After receiving the early warning information, the relevant staff of the management end can promptly understand the serious pollution or danger of floating garbage at sea, so as to quickly formulate and implement targeted cleaning and governance measures.

[0041] S104, remotely controlling the flight parameters of the drone to salvage garbage according to the marine garbage distribution map.

[0042] The marine debris identification system issues instructions to the drone based on the generated marine debris distribution map and remotely controls its flight parameters to achieve accurate garbage salvage operations. The flight control module in the system is responsible for parsing the information in the marine debris distribution map and converting it into flight instructions that the drone can understand. First, based on the location of the marine debris marked on the distribution map, the flight control module plans the optimal flight path for the drone. The path planning algorithm takes into account a variety of factors, such as the current location of the drone, the density of marine debris distribution, marine environmental conditions (such as wind speed, water flow direction, etc.), and the drone's endurance.

[0043] In order to ensure that the drone can accurately reach the target garbage area, the flight control module adjusts the drone's flight attitude and speed in real time. The drone is equipped with a variety of sensors, such as inertial measurement units (IMUs), barometric altimeters, GPS receivers, etc. These sensors provide real-time feedback on the drone's position, altitude, attitude and other information. Based on these feedback data, the flight control module uses algorithms such as PID (proportional-integral-differential) controllers to accurately adjust the drone's propeller speed and rudder angle to keep the drone on the predetermined flight path.

[0044] When approaching the target area of ​​floating garbage at sea, the flight control module reduces the flight speed of the drone and adjusts its altitude at the same time to more accurately identify and grab the garbage. The visual recognition system carried by the drone plays an important role in this stage. The visual recognition system reconfirms the location and type of floating garbage at sea by analyzing the real-time images to ensure the accuracy of grabbing. When the drone reaches the appropriate location, the flight control module controls the mechanical arm or garbage grabbing device to perform garbage salvage operations. The movement of the mechanical arm is controlled by the system's pre-set program. According to the shape, size and location of the floating garbage at sea, the appropriate grabbing method, such as clamping and adsorption, is selected. During the garbage salvage process, the flight control module continuously monitors the status and load of the drone. If the center of gravity of the drone changes due to grabbing garbage, the flight control module will adjust the flight parameters in time to maintain the stability of the drone. At the same time, the system transmits the real-time situation of garbage salvage back to the monitoring terminal, and the management personnel can remotely intervene in the operation of the drone according to the actual situation. After completing the garbage salvage mission, the flight control module controls the drone to return to the designated location according to the preset return path and properly dispose of the salvaged garbage.

[0045] In the embodiment of the present application, a series of technical means are used to control the salvage of drones based on the distribution map, by using a drone equipped with a combination of multiple cameras and multiple sensors to obtain sea surface image information, and using a marine debris identification model constructed using deep learning to determine the debris information, and then combining the positioning information to generate a distribution map. This makes it possible to comprehensively and accurately handle the problem of marine debris in complex marine environments and weather conditions.

[0046] After combining the above content, the following is a more specific process description of the method provided in this embodiment. Please refer to Figure 2 , which is another process schematic diagram of the method for identifying marine floating garbage based on drones in the embodiments of this application.

[0047] S201. Obtain multiple water flow information and wind direction information through sensors. The water flow information includes water flow speed and direction data, and the wind direction information includes wind direction and wind speed data; The system deploys water flow sensors on drones and marine monitoring buoys. The acoustic Doppler current profiler is one of the commonly used water flow sensors. It emits sound waves into the water body and measures the flow velocity and direction of different depth water layers according to the Doppler frequency shift reflected by the sound waves after encountering particulate matter in the water, and can accurately obtain water flow speed and direction data, with a measurement accuracy reaching the centimeter / second level. The electromagnetic current meter uses the principle of electromagnetic induction. When water flows through a magnetic field, an induced electromotive force will be generated, and the water flow speed is calculated by measuring this electromotive force. This sensor has high stability and is relatively less affected by the external environment, and can work stably for a long time in a complex marine environment. These sensors are distributed at a certain interval in the monitoring area, can monitor water flow information in real time, and transmit the data back to the marine floating garbage identification system through a wireless communication module.

[0048] In terms of obtaining wind direction information, the system uses an ultrasonic wind speed and direction sensor. This instrument calculates the wind speed by measuring the time difference of ultrasonic waves propagating in the air along the wind direction and against the wind direction, and determines the wind direction according to the difference in ultrasonic wave propagation time in different directions. Its response speed is fast, and it can quickly capture changes in wind direction and wind speed, providing real-time and accurate wind direction information for the system. At the same time, multiple such instruments are arranged on the flight path of the drone and at key monitoring points to build a wind direction monitoring network to ensure the comprehensiveness and accuracy of the data.

[0049] S202. According to the water flow information, the wind direction information, and the type of marine floating garbage, combine the water flow model and the wind direction model to determine the drift path information of the marine floating garbage; The system adopts a water flow model constructed based on the principles of fluid mechanics, such as the Navier-Stokes equation model discretized by the finite volume method. This model divides the ocean area into multiple tiny control volumes, and calculates the water flow velocity and pressure distribution by solving the mass and momentum conservation equations within each control volume. The model takes into account the viscosity and density of seawater, as well as the influence of factors such as seabed topography and coastline shape on the water flow. In the area near the coastline, due to seabed friction and terrain changes, the water flow velocity and direction will change. The model improves the accuracy of water flow simulation by accurately simulating these boundary conditions. The wind direction model uses the principles of atmospheric dynamics and is based on the numerical simulation model of the wind field, considering the influence of factors such as atmospheric pressure gradient, Coriolis force, and friction on the wind direction and wind speed. By analyzing meteorological satellite data, ground meteorological station observation data, and the output results of numerical weather prediction models, wind field information at different heights and regions is obtained and input into the wind direction model for calculation. The system uses a high-performance computing cluster to run the water flow model and the wind direction model, and processes the obtained water flow information, wind direction information, and data on the types of marine debris.

[0050] After inputting the water flow information, wind direction information, and data on the types of marine debris into the verified and optimized water flow model and wind direction model, the models simulate the movement trajectory of marine debris under the action of water flow and wind through complex calculations. The system takes the initial position of the marine debris as the starting point, and according to a certain time step, calculates the displacement of the marine debris within each time step obtained from the model calculation, and successively calculates the position coordinates of the marine debris at different times, so as to determine its drift path information. The system can also perform visualization processing on the drift path information. Through geographic information system (GIS) technology, the drift path of the marine debris is displayed on an electronic nautical chart or a satellite image base map, providing intuitive and accurate information for marine environmental protection and management departments, so as to plan the cleaning work in advance and improve the cleaning efficiency.

[0051] S203. Obtain the flight time information of the unmanned aerial vehicle flying to the position of the marine debris; The system estimates the flight time by combining the current position of the drone, the position of the target garbage, and the preset flight path, and referring to the conventional flight speed of the drone. In the actual marine environment, factors such as wind speed, wind direction, and sea waves will significantly affect the flight of the drone. Headwinds will reduce the flight speed of the drone, and rough sea waves will interfere with flight stability, resulting in speed changes. Therefore, the system uses the wind speed sensor, sea wave monitoring equipment, etc. carried by the drone to monitor the environmental data in real time, and adjusts the estimated flight time based on this data. In order to obtain more accurate flight time information, the sensors of the drone itself play an important role. The GPS real-time feedbacks the position change of the drone, and the inertial measurement unit monitors the flight attitude and acceleration. The system analyzes these sensor data to grasp the flight state of the drone in real time. Once it is found that the flight state is abnormal and causes the speed to change, it will recalculate the flight time in time.

[0052] S204. Determine the position change of the marine drifting garbage within the flight time information according to the drift path information; The marine drifting garbage recognition system determines the position change of the marine drifting garbage within the flight time based on the drift path information and flight time information of the marine drifting garbage. The system divides the flight time into multiple shorter time periods. Within each time period, according to the current water flow speed, wind direction and speed, and the type characteristics of the marine drifting garbage, it speculates the moving distance and direction of the marine drifting garbage. Lighter garbage such as plastic bottles has different moving speeds and directions from heavier garbage such as metal cans under the same water flow and wind conditions. The moving directions and distances of specific different types of marine drifting garbage can be determined as follows: establish a moving trajectory analysis model. The marine drifting garbage recognition system needs to collect a large amount of moving trajectory data of different types of marine drifting garbage under different wind directions and speeds and water flow speeds. The system obtains data through experiments. In an experimental pool or a specific sea area, different types of marine drifting garbage are put in, such as plastic bottles, metal cans, plastic bags, etc., and different water flow speeds and wind direction and speed conditions are set at the same time. Using high-precision positioning equipment, such as differential GPS, the position change of the marine drifting garbage is recorded in real time, so as to obtain its moving trajectory data. Use machine learning algorithms to construct a moving trajectory analysis model. The recurrent neural network (RNN) and its variant long short-term memory network (LSTM) in deep learning are commonly used algorithms. These algorithms can process moving trajectory data with time series characteristics, capture the complex relationship between the moving trajectory of marine drifting garbage and wind direction and speed, water flow speed, and garbage type. In the model structure design, the type of marine drifting garbage, wind direction and speed, and water flow speed are used as input features. For the type of marine drifting garbage, one-hot encoding or word vectors are used for feature representation, so that the model can distinguish different types of garbage. The wind direction and speed and water flow speed are directly input into the model as numerical features. The output of the model is the position coordinate change of the marine drifting garbage in the future period of time, that is, the moving direction and distance.

[0053] The system can also take into account other situations that floating garbage may encounter during its movement, such as collisions with other objects in the sea or the impact of marine life. To this end, the system can build a collision and interference simulation model. Specifically, by collecting distribution data of common objects in the ocean and the activity patterns of marine life, the system can predict the impact of these factors on the movement of floating garbage to a certain extent. When the simulation model determines that floating garbage may collide, the system adjusts the movement direction and speed of the floating garbage according to the material, shape of the collision object and the movement state of the floating garbage, and recalculates its position change.

[0054] S205, determining the final position of the marine debris when the drone arrives at the marine debris position according to the position change; The marine debris identification system determines the final location of the marine debris when the drone arrives based on the changes in its location during the flight time. The system calculates the location changes in each time period, finds the location of the marine debris at the time of the drone's arrival, and determines it as the final location. After determining the final location, the system is not fixed. Due to the ever-changing marine environment, the system will dynamically adjust the final location based on the latest information such as water flow and wind direction. If the water flow speed suddenly increases or the wind direction changes during the drone's flight, the system will recalculate the location changes of the marine debris based on the new environmental data, update the final location, and ensure that the drone can accurately find the marine debris.

[0055] S206: Control the drone to fly directly to the final location to salvage the garbage.

[0056] After the marine debris identification system determines the final location of marine debris, it controls the drone to fly directly to that location for garbage salvage. The flight control module plans the flight path of the drone based on the final location information. During the planning process, the module comprehensively considers the drone's current location, final location, marine environmental conditions, and the drone's own performance parameters, such as endurance and maximum flight speed, and generates the optimal flight path through an intelligent path planning algorithm. When the drone approaches the final location of marine debris, the flight control module reduces the flight speed and adjusts the altitude to more accurately identify and grab the debris.

[0057] In the embodiment of the present application, sensors are deployed on drones and sea surface monitoring buoys to obtain water flow and wind direction information, and the drift path of the floating garbage at sea is determined by combining the water flow and wind direction models. The flight time of the drone is accurately obtained and the final location of the floating garbage at sea is determined accordingly. Finally, the drone is controlled to salvage the floating garbage, thereby achieving efficient and accurate control of the entire process from monitoring to salvaging the floating garbage at sea. This not only solves the problem of difficulty in accurately tracking the location of garbage and low salvage efficiency in traditional floating garbage treatment, but also improves the success rate of drone salvage and reduces energy consumption.

[0058] The following describes the sea drift garbage recognition system in the embodiments of the present invention application from the perspective of hardware processing. Please refer to Figure 3 , which is a schematic structural diagram of an entity device of the sea drift garbage recognition system in the embodiments of the present application.

[0059] It should be noted that Figure 3 The structure of the sea drift garbage recognition system shown is only an example, and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0060] As Figure 3 shown, the sea drift garbage recognition system includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the Read-Only Memory (ROM) 302 or the program loaded from the storage section 308 into the Random Access Memory (RAM) 303, such as executing the method described in the above embodiments. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus 304. The Input / Output (I / O) interface 305 is also connected to the bus 304.

[0061] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a Liquid Crystal Display (LCD), an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed, so that the computer program read from it can be installed into the storage section 308 as needed.

[0062] In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are executed.

[0063] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.

[0064] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings.

[0065] Specifically, the sea drift garbage recognition system of this embodiment includes a processor and a memory, and a computer program is stored on the memory. When the computer program is executed by the processor, the sea drift garbage recognition method based on an unmanned aerial vehicle provided in the above embodiment is implemented.

[0066] As another aspect, the present invention also provides a computer-readable storage medium, which may be included in the sea drift garbage recognition system described in the above embodiments; or may exist alone without being assembled into the sea drift garbage recognition system. The above storage medium carries one or more computer programs, and when the one or more computer programs are executed by a processor of the sea drift garbage recognition system, the sea drift garbage recognition system implements the method for recognizing sea drift garbage based on an unmanned aerial vehicle provided in the above embodiments.

[0067] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0068] As used in the above embodiments, depending on the context, the term "when..." can be interpreted to mean "if...", or "after...", or "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if detecting (the stated condition or event)" can be interpreted to mean "if determining...", or "in response to determining...", or "when detecting (the stated condition or event)", or "in response to detecting (the stated condition or event)".

[0069] Those of ordinary skill in the art can understand all or part of the processes in the methods of the above embodiments. These processes can be completed by relevant hardware instructed by a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The foregoing storage medium includes: various media such as ROM or random access memory RAM, magnetic disk, or optical disc that can store program codes.

Claims

1. A method for identifying marine debris based on drones, applied to a marine debris identification system, characterized in that: The method comprises: Acquire multiple sea surface image information through drones; Inputting the sea surface image information into a marine debris identification model to determine marine debris information, wherein the marine debris identification model is constructed by deep learning from a plurality of historical sea surface image information sets that have been manually labeled with marine debris information, and the marine debris information includes at least the amount and type of marine debris; generating a distribution map of floating garbage at sea by combining the floating garbage at sea location information and the floating garbage at sea information, wherein the floating garbage at sea location information is obtained according to the positioning module; The flight parameters of the remote-controlled drone are used to salvage garbage according to the marine garbage distribution map.

2. The method according to claim 1, characterized in that Before the step of obtaining multiple sea surface image information through a drone, it also includes: Obtain light intensity information through multiple sensors; In combination with the light intensity information, the camera parameters of the drone are adjusted according to a preset parameter adjustment strategy library; If the light intensity information is greater than a preset strong light intensity threshold, the aperture size of the drone camera is reduced and the sensitivity is lowered according to the preset strong light amplitude; If the light intensity information is less than a preset low-light intensity threshold, the aperture of the drone camera is increased and the sensitivity is improved according to the preset low-light amplitude.

3. The method according to claim 2, characterized in that If the light intensity information is less than the preset low light intensity threshold, after the step of increasing the aperture of the drone camera and improving the sensitivity according to the preset low light amplitude, the method further includes: Combining the plurality of sea surface image information with a preset meteorological recognition model to determine real-time sea environment status information; Rating the marine environment information in combination with the preset marine safety level to determine the current marine safety level; If the current sea area safety level is greater than the preset safety threshold, an evacuation command is sent to the drone.

4. The method according to claim 1, characterized in that: After the step of combining the marine debris location information and the marine debris information to generate a marine debris distribution map, the method further includes: Obtain summary information on the amount of marine debris and the types of marine debris in multiple target areas; If the summary information of the amount of marine debris exceeds a preset threshold or the type of marine debris belongs to a preset danger level, an early warning message is sent to a preset management terminal.

5. The method according to claim 1, characterized in that After the step of combining the marine debris location information and the marine debris information to generate a marine debris distribution map, the method further includes: Acquire multiple water flow information and wind direction information through sensors, wherein the water flow information includes water flow speed and direction data, and the wind direction information includes wind direction and wind speed data; According to the water flow information, the wind direction information and the type of marine debris, the drift path information of the marine debris is determined in combination with the water flow model and the wind direction model.

6. The method according to claim 5, characterized in that After the step of determining the drift path information of the marine floating garbage according to the water flow information, the wind direction information and the type of marine floating garbage in combination with the water flow model and the wind direction model, the method further includes: Obtain the flight time information of the drone flying to the location of the marine debris; Determine the position change of the marine floating garbage within the flight time information according to the drift path information; Determine the final position of the floating garbage at sea when the drone arrives at the floating garbage position according to the position change; The drone is controlled to fly directly to the final location for garbage salvage.

7. The method according to claim 1, characterized in that After the step of obtaining multiple sea surface image information through the drone, it also includes: An image preprocessing module is used to perform preprocessing operations on the sea surface image information, and the preprocessing operations include grayscale processing, noise reduction processing and size normalization processing.

8. A marine debris identification system, characterized in that: The marine debris identification system comprises: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions to enable the marine debris identification system to execute the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a marine debris identification system, the marine debris identification system is caused to execute the method as claimed in any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on a marine debris identification system, the marine debris identification system is enabled to execute the method according to any one of claims 1 to 7.

Images