Intelligent detection method and system for self-adaptive sea surface garbage, medium and program product

Abstract

The invention provides a self-adaptive sea surface garbage intelligent detection method and system, a medium and a program product, and relates to the technical field of sea surface garbage intelligent detection. The method comprises the following steps: collecting sea surface infrared thermogram information by using a plurality of thermal infrared imagers, inputting the information into an offshore garbage identification model which is trained by using an infrared thermogram sample with garbage labels in advance, determining suspected offshore garbage information and obtaining positioning of the suspected offshore garbage information; the method comprises the following steps: firstly, obtaining suspected seaborne garbage, then obtaining motion track information of the suspected seaborne garbage in a set time, then determining final seaborne garbage information in combination with the track information, and finally sending the final seaborne garbage information and garbage positioning information to a management end to realize seaborne garbage monitoring and information transmission. By implementing the method, sea surface garbage can be accurately detected under complex meteorological conditions.

Description

Technical Field

[0001] This application relates to the technical field of intelligent detection of marine litter, and particularly to an intelligent detection method, system, medium, and program product for adaptive marine litter. Background Art

[0002] With the rapid development of the global economy and the large-scale development and utilization of marine resources, the marine environment is facing unprecedented pressure, and the problem of marine litter pollution is becoming increasingly prominent. Marine litter not only destroys the marine ecological balance and threatens the survival of marine organisms, but also has a negative impact on maritime navigation safety, coastal tourism, etc. Therefore, timely and accurate detection of marine litter is crucial for marine environmental protection and the sustainable development of related industries.

[0003] Currently, traditional marine litter detection technologies mainly rely on fixed cameras to collect video stream information of the sea surface. These fixed cameras are installed along the coast, on islands, etc., and continuously shoot the sea surface to obtain continuous video images. At the same time, technicians pre-train an image classification model using a large amount of image data containing various types of marine litter. During actual detection, each frame of the video stream collected is input into the pre-trained image classification model. The model analyzes and judges the objects in the image based on the previously learned litter features, such as the shape, color, texture, etc. of the litter, attempts to identify the marine litter in it, and determines information such as the location of the litter.

[0004] However, when facing special conditions such as cloudy days or thick fog, the lighting conditions are extremely poor, the detailed features of the litter are difficult to clearly present, and a large amount of feature information such as shape and color relied on by the image classification model is missing. Therefore, it is difficult to accurately detect marine litter under complex meteorological conditions. Summary of the Invention

[0005] This application provides an intelligent detection method, system, medium, and program product for adaptive marine litter, which is used to efficiently and accurately solve the problem of marine litter monitoring under special meteorological conditions.

[0006] In a first aspect, this application provides an intelligent detection method for adaptive marine litter, which is applied to an intelligent detection system. The method includes: obtaining sea surface infrared thermal map information through multiple infrared thermal imagers; combining the sea surface infrared thermal map information, determining suspected marine litter information through a marine litter recognition model, and obtaining litter positioning information, where the marine litter recognition model is obtained through deep learning training in advance according to multiple infrared thermal map samples of the sea surface with litter annotations; obtaining the movement trajectory information of the suspected marine litter information within a set time; determining the final marine litter information in combination with the movement trajectory information; and sending the final marine litter information and the litter positioning information to the management terminal.

[0007] By adopting the above technical solutions, multiple infrared thermal imagers can obtain the infrared thermal map information of the sea surface, making up for the defect that traditional fixed cameras cannot clearly present the detailed features of garbage under poor lighting conditions on cloudy days. The marine garbage recognition model is trained based on deep learning and can analyze the infrared thermal map information. By combining the thermal map information with the model, the suspected marine garbage information and its location are determined, realizing the preliminary detection and location of marine garbage, providing basic data for subsequent precise garbage processing, improving the feasibility of detecting marine garbage in complex environments, and being able to more effectively detect marine garbage compared with traditional detection technologies.

[0008] Combined with some embodiments of the first aspect, in some embodiments, the step of determining the suspected marine garbage information and obtaining the garbage location information by combining the infrared thermal map information of the sea surface specifically includes: performing a preprocessing operation on the infrared thermal map information of the sea surface, and the preprocessing operation at least includes removing noise interference and enhancing image contrast; smoothing the random noise points in the infrared thermal map information of the sea surface through an image filtering algorithm; stretching the image gray range in the infrared thermal map information of the sea surface by using histogram equalization technology; using the marine garbage recognition model to extract features from the infrared thermal map information of the sea surface to determine the shape, size and temperature distribution data of the thermal radiation emitted by the infrared thermal imager; comparing the shape, size and temperature distribution data of the thermal radiation with the garbage feature library in the marine garbage recognition model to determine the similarity data; if the similarity data exceeds the set similarity threshold, then determine the suspected marine garbage information.

[0009] By adopting the above technical solutions, the preprocessed image enables the model to better extract features, reduces misjudgment caused by image quality problems, improves the accuracy of determining the suspected marine garbage information, and thus enhances the reliability of the entire detection process.

[0010] Combined with some embodiments of the first aspect, in some embodiments, the step of obtaining the movement trajectory information of the suspected marine garbage within a set time specifically includes: obtaining the infrared thermal map information of the suspected garbage area by multiple infrared thermal imagers at set time intervals; performing a preprocessing operation on the infrared thermal map information of the suspected garbage area, and the preprocessing operation at least includes noise reduction and histogram equalization; using the optical flow method to track the suspected garbage in the infrared thermal map information of the sea surface, and combining the garbage location information of the suspected garbage at different times, connecting the position coordinates of each time point in sequence to determine the movement trajectory information; using the Kalman filtering algorithm to optimize the movement trajectory information.

[0011] By adopting the above technical solutions, the determined movement trajectory can be made more accurate, effectively analyzing the movement law of the suspected garbage, providing a strong basis for judging whether it is real garbage, and enhancing the credibility of the detection result.

[0012] In some embodiments in combination with some embodiments of the first aspect, in the step of determining the final marine litter information in combination with the movement trajectory information, it specifically includes: in combination with the movement trajectory information, determining whether the suspected marine litter information corresponding to the movement trajectory information is a living being through a biological movement recognition model, and the biological movement recognition model is obtained through deep learning training in advance based on a plurality of labeled marine biological and litter movement trajectory samples; if it is not a living being, determining that the suspected marine litter information is the final marine litter information.

[0013] By adopting the above technical solution, with the aid of the biological movement recognition model, it is determined whether the suspected marine litter is a living being based on the movement trajectory. This model is trained based on a large number of labeled samples and has a high recognition ability. If it is determined that it is not a living being, it can be determined as the final marine litter information, which can eliminate the interference of marine organisms on litter detection, avoid misjudging living beings as litter, significantly improve the accuracy of determining the final marine litter information, ensure that the detection result truly reflects the situation of marine litter, and reduce incorrect operations in subsequent processing.

[0014] In some embodiments in combination with some embodiments of the first aspect, after the step of determining whether the suspected marine litter information corresponding to the movement trajectory information is a living being through a biological movement recognition model in combination with the movement trajectory information, it further includes: if it is impossible to determine whether the suspected marine litter information corresponding to the movement trajectory information is a living being through the biological movement recognition model, obtaining the thermal signal intensity data of the suspected marine litter at a set time; in combination with the thermal signal intensity data, determining the thermal signal similarity through a marine biological metabolic thermal signal intensity change model, and the marine biological metabolic thermal signal intensity change model is constructed through deep learning in advance based on the thermal signal intensity change samples of a plurality of marine organisms in different physiological states; if the thermal signal similarity is lower than the set biological characteristic threshold, determining that the suspected marine litter information is the final marine litter information.

[0015] By adopting the above technical solution, when the biological movement recognition model is unable to determine, the thermal signal intensity data of the suspected marine litter is obtained, and the thermal signal similarity is determined in combination with the marine biological metabolic thermal signal intensity change model. This model is constructed based on the thermal signal change samples of marine organisms and can further distinguish from the perspective of thermal signals. If the similarity is lower than the threshold, it is determined as the final litter, which provides a supplementary means for judgment, solves the uncertainty of movement trajectory judgment, further improves the ability to accurately identify litter in complex situations, and improves the litter detection mechanism.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of obtaining the infrared thermal image information of the sea surface through multiple infrared thermal imagers, the method further includes: analyzing the image quality information of the infrared thermal image information of the sea surface by an image analysis module, where the image information at least includes the sharpness and contrast of the image edges; if the image quality information deteriorates and the downward trend conforms to the characteristics of the lens being blocked, it is determined that the lens is blocked by sea spray or impurities; controlling a micro jet device arranged around the lens to spray high-pressure cleaning gas towards the lens.

[0017] By adopting the above technical solution, the state of the lens can be monitored in real time, and it can be cleaned in time when the lens is blocked by sea spray or impurities, ensuring that the infrared thermal imager continuously obtains high-quality thermal images, avoiding inaccurate detection data caused by lens blockage, and ensuring the stable and reliable progress of the detection work.

[0018] In combination with some embodiments of the first aspect, in some embodiments, after the step of sending the final marine garbage information and the garbage positioning information to the management terminal, the method further includes: monitoring the operating state of the infrared thermal imager in real time, where the operating state at least includes the power and signal strength of the device; when any one of the parameters in the operating state exceeds the parameter standard range, a maintenance reminder message is automatically generated and pushed to the management terminal.

[0019] By adopting the above technical solution, when the parameter exceeds the standard range, a maintenance reminder message is automatically generated and pushed to the management terminal, the device anomaly can be discovered in time, the management terminal can respond quickly and arrange maintenance work, avoiding the influence of insufficient device power or signal problems on data collection and transmission, ensuring the stable operation of the infrared thermal imager, maintaining the normal operation of the entire detection system, and ensuring the continuity and accuracy of the sea surface garbage detection work.

[0020] In a second aspect, the present application provides a server, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the server to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0021] In a third aspect, the present application provides a computer-readable storage medium, including instructions, when the instructions run on the server, causing the server to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0022] In a fourth aspect, the present application provides a computer program product, when the computer program product runs on the server, causing the server to execute the method described in the first aspect and any possible implementation manner in the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Since the technical means of using multiple infrared thermal imagers to obtain infrared thermal map information of the sea surface and combining a sea garbage recognition model trained by deep learning to determine suspected garbage information and positioning information are adopted, the technical problem that it is difficult to clearly detect sea garbage under insufficient light in the prior art is effectively solved. Furthermore, the technical effect of efficiently and accurately preliminarily detecting and positioning sea garbage in a complex environment is achieved.

[0024] 2. Since the technical means of using multiple infrared thermal imagers to obtain thermal map information of suspected garbage areas at time intervals and combining preprocessing, optical flow method tracking, and Kalman filtering algorithm to optimize the movement trajectory are adopted, the technical problem that it is difficult to accurately track the movement trajectory of sea garbage in the prior art is effectively solved. Furthermore, the technical effect of more accurately grasping the movement law of suspected sea garbage, providing a strong basis for subsequent judgment, and enhancing the credibility of detection results is achieved.

[0025] 3. Since the technical means of combining a marine biological metabolic heat signal intensity change model when the biological movement recognition model cannot judge and determining the final sea garbage information based on the similarity of thermal signals are adopted, the technical problem that it is difficult to accurately distinguish marine organisms and sea garbage only by the movement trajectory is effectively solved. Furthermore, the technical effect of improving the garbage recognition mechanism from the perspective of thermal signals and enhancing the ability to accurately identify sea garbage in a complex situation is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of an intelligent detection method for adaptive sea garbage in the embodiments of the present application; Figure 2 is another flowchart of an intelligent detection method for adaptive sea garbage in the embodiments of the present application; Figure 3 is a schematic structural diagram of an entity device of an intelligent detection system in the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "an", "the", "above", "said", "this" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more of the listed items.

[0028] Hereinafter, the terms "first" and "second" are for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0029] In the marine environment, there are various organisms that move on or near the sea surface. These organisms are easily confused with marine litter in terms of appearance or motion state on cloudy days, causing interference to the marine litter detection work. From the perspective of appearance, the forms, colors, and floating states of some organisms are similar to those of litter. For example, when jellyfish float on the sea surface, their transparent or semi-transparent bodies and irregular shapes may be visually indistinguishable from some lightweight plastic litter such as plastic bags and plastic films. When seabird feathers fall off and float on the sea surface, their colors and forms may be similar to paper litter. Analyzing from the aspect of motion state, the motion modes of some organisms have similar characteristics to the motion of litter under the action of sea waves and ocean currents. When small fish groups forage or swim on the sea surface, their collective motion trajectories may show a relatively random and irregular state, which is similar to the motion trajectories of some litter under complex sea conditions. In this solution, it is specifically pointed out that organisms refer to those that will be mistaken for marine litter on or near the sea surface, clarifying the application objects of the organism motion recognition model and the model of the change in the intensity of the metabolic heat signal of marine organisms. This enables the intelligent detection system to accurately distinguish such easily confused organisms and litter, improve the accuracy of marine litter detection, reduce the occurrence of misjudgments, and thus ensure the scientificity and reliability of the marine litter detection work, providing more accurate data support for marine environmental protection and litter cleaning work.

[0030] For ease of understanding, the method provided in this embodiment will be described in terms of its process below. Please refer to Figure 1 , which is a schematic flowchart of a process of the intelligent detection method for adaptive marine litter in the embodiments of the present application.

[0031] S101. Obtain the infrared thermal image information of the sea surface through multiple infrared thermal imagers; The intelligent detection system first obtains the current sea surface illumination intensity information through a light sensor on a drone or an observation site. If this illumination intensity information is greater than the set illumination intensity threshold, it means that the current illumination conditions are good and can meet the shooting requirements of the camera. In this case, the system will enable the camera to take pictures of sea surface garbage, and these captured pictures will be transmitted to a dedicated image recognition module, which will use a pre-trained image recognition algorithm to analyze and judge the objects in the pictures. If this illumination intensity information is less than the set illumination intensity threshold, it indicates that the current illumination conditions are poor and it is not suitable to use the camera to take pictures of sea surface garbage, because in such a low-light environment, the pictures taken often have problems such as difficult to clearly present the detailed features of the garbage, and the quality of the images will be greatly reduced, which will in turn cause the image recognition module to be unable to accurately identify the garbage, resulting in a large number of misjudgments or missed judgments. In this case, multiple infrared thermal imagers are used to obtain the infrared thermal map information of the sea surface, and these infrared thermal imagers can detect the infrared radiation emitted by objects and convert it into a thermal image.

[0032] To ensure obtaining a comprehensive and accurate infrared thermal map, multiple infrared thermal imagers will be set at different positions and angles at the observation site, or infrared thermal imagers will be set on the drone, so as to cover a wider sea area and reduce the occurrence of blind spots. During the process of obtaining the infrared thermal map information, some technical means will also be taken to improve the image quality. For example, using zoom thermal imaging technology, by changing the focal length of the lens, clear observation of targets at different distances can be achieved without changing the lens or moving the device position. Its internally integrated advanced image processing system can perform real-time processing on the collected infrared images, including enhancing image contrast, removing noise, etc., so as to obtain a clearer and more recognizable image.

[0033] In some embodiments, to ensure the quality of the infrared thermal image information of the sea surface obtained by the infrared thermal imager on the unmanned aerial vehicle, the system is provided with an image analysis module. This module analyzes the image quality information of the infrared thermal image information of the sea surface. Among them, the image edge sharpness and contrast are the key analysis indicators. The image analysis module uses professional image processing algorithms and technologies to evaluate the infrared thermal image of the sea surface. For the image edge sharpness, it is measured by detecting the sharpness of the edges of objects in the image. Under ideal conditions, the edges of various objects in the infrared thermal image should be clearly distinguishable, such as the edges of waves, the edges of possible garbage, etc. If the edges become blurred, it means that the image edge sharpness has decreased. The contrast reflects the brightness difference between different regions in the image. In an image with high contrast, the difference between the bright area and the dark area is obvious, and the details of the object can be presented more clearly; on the contrary, in an image with low contrast, it is difficult to distinguish the object from the background. When the image analysis module determines that the image quality information has decreased, it will further determine whether this decreasing trend conforms to the characteristics of the lens being blocked. When the lens is blocked by sea spray or impurities, the image quality decreases in a specific pattern. When sea spray adheres to the lens, the whole image becomes blurred, the edge sharpness decreases, and at the same time, due to the scattering and absorption of infrared radiation by the spray, the image contrast decreases. When impurities block the lens, it may cause the loss or blurring of the image information in a local area, affecting the integrity and sharpness of the image. The system compares the current pattern of image quality degradation with the pre-set image feature template when the lens is blocked. If the two match, it can be determined that the lens is blocked by sea spray or impurities. Once it is determined that the lens is blocked, the intelligent detection system will immediately control the operation of the micro jet devices arranged around the lens. These micro jet devices are pre-installed around the infrared thermal imager lens to ensure that the high-pressure cleaning gas ejected can cover the entire lens surface. When receiving the instruction sent by the system, the micro jet devices quickly eject high-pressure cleaning gas towards the lens. The powerful impact force generated by the high-pressure gas can effectively blow off the sea spray and impurities on the lens surface, making the lens return to a clean state. This process can be completed quickly and automatically without manual intervention, ensuring that the infrared thermal imager continuously obtains high-quality infrared thermal image information of the sea surface, ensuring that the detection of marine garbage is not affected by lens blockage and proceeding stably and reliably.

[0034] S102. Combine the infrared thermal image information of the sea surface, determine the suspected marine garbage information through the marine garbage recognition model, and obtain the garbage positioning information. The marine garbage recognition model is obtained through deep learning training in advance based on multiple infrared thermal image samples of the sea surface with garbage annotations; The intelligent detection system first performs preprocessing operations on the obtained sea surface infrared thermal image information, which may include using techniques to remove noise interference, such as adopting a strip noise elimination method based on a convolutional neural network to improve the imaging accuracy and uninterrupted monitoring performance of infrared thermal imaging; or applying image filtering algorithms, such as mean filtering, median filtering, etc., to smooth the random noise points in the sea surface infrared thermal image information. At the same time, the histogram equalization technique is used to stretch the image gray range and enhance the image contrast, making the details in the thermal image more obvious. Then, a pre-trained deep learning-based marine litter recognition model is used to extract features from the preprocessed sea surface infrared thermal image information. This model is trained based on a large number of marine litter infrared thermal image samples with litter annotations, and it can determine data such as the shape, size, and temperature distribution of the thermal radiation emitted by the infrared thermal imager.

[0035] After feature extraction, the thermal radiation shape, size, and temperature distribution data are compared with the litter feature library in the marine litter recognition model. By calculating the similarity data, it is determined whether the detected thermal image information is similar to the features in the litter feature library. If the similarity data exceeds the set similarity threshold, it can be determined that there is suspected marine litter information in this area. At the same time, by combining the position information of multiple infrared thermal imagers and the relative positions in the thermal image, the positioning information of the litter can be obtained. For example, when performing similarity comparison, feature matching algorithms in deep learning can be used. These algorithms can automatically learn and extract key features in the image and match and compare them with the features in the litter feature library. In addition, to further improve the detection accuracy, some local peak detection methods may be adopted. First, the average gray correction method is used to improve the contrast of the target and the quality of the infrared image, then the local area is cropped to improve the target detection efficiency and detection rate, and then the eight-connected domain peak point detection is performed on the local image. The pseudo-targets are removed by screening according to the height of the peak points, and finally, some pseudo-targets are further removed according to the contrast analysis to obtain the final detection result.

[0036] When obtaining the positioning information of the suspected marine litter, the relative position relationship between multiple infrared thermal imagers can be used, and through triangulation or other related positioning algorithms, the position of the suspected marine litter on the sea surface can be accurately determined. In this way, the intelligent detection system can combine the sea surface infrared thermal image information and the pre-trained deep learning-based marine litter recognition model to accurately determine the suspected marine litter information and obtain its corresponding positioning information.

[0037] S103. Obtain the movement trajectory information of the suspected marine litter information within a set time; The intelligent detection system obtains the movement trajectory information of suspected marine garbage within a set time through a series of precise and orderly operations. First, multiple infrared thermal imagers continuously obtain infrared thermal image information of suspected garbage areas at a pre-set time interval. The setting of this time interval needs to take into account many factors, such as the approximate movement speed of sea surface garbage, detection accuracy requirements, and the system's computing and processing capabilities. If the time interval is too long, key position changes in the garbage movement process may be missed; if it is too short, a large amount of data will be generated, increasing the system's storage and processing burden.

[0038] After obtaining the infrared thermal image information of the suspected garbage area, the intelligent detection system will pre-process this information and use advanced noise reduction algorithms, such as the noise reduction method based on wavelet transform. This method can effectively remove noise interference in the infrared thermal image, while retaining the detailed information of the image as much as possible, so that the image can more clearly and accurately reflect the actual situation of the suspected garbage. In addition, the histogram equalization technology can also be used to adjust the grayscale distribution of the image, stretch the grayscale range of the image, enhance the contrast of the image, and further highlight the characteristics of the suspected garbage, which is convenient for subsequent analysis and processing. The optical flow method is then used to track the suspected garbage in the infrared thermal image information of the sea surface. The basic principle of the optical flow method is based on the assumption that the brightness of the pixel points in the image remains unchanged in a short period of time. By calculating the motion vector of the pixel points in two adjacent frames of the image, the movement of the suspected garbage is determined. The intelligent detection system combines the garbage location information of the suspected garbage at different times, connects the position coordinates of each time point in sequence, and preliminarily determines the movement trajectory information of the suspected marine garbage.

[0039] However, due to the complex and changeable actual marine environment, the presence of interference factors such as waves and sea breezes, the initially determined motion trajectory may have certain errors. In order to improve the accuracy and stability of the motion trajectory, the intelligent detection system can use the Kalman filter algorithm to optimize the motion trajectory information. The Kalman filter algorithm is an algorithm that uses the linear system state equation to optimally estimate the system state through system input and output observation data. In this scenario, it can predict and correct the position, speed and other states of suspected marine garbage based on the existing motion trajectory information and the currently acquired infrared thermal map data. For example, when waves cause abnormal fluctuations in the suspected garbage position in the infrared thermal map, the Kalman filter algorithm can smooth the abnormal data based on previous movement trends and model predictions, so that the final motion trajectory is more in line with the actual movement law of suspected marine garbage, providing a more reliable basis for subsequent judgments.

[0040] S104, determining final marine garbage information based on the movement trajectory information; The intelligent detection system uses a biological motion recognition model to judge the suspected marine garbage information corresponding to the motion trajectory information and determine whether it is a living being. This biological motion recognition model is constructed in advance through deep learning training based on multiple labeled marine biological and garbage motion trajectory samples. The model has powerful pattern recognition capabilities and can distinguish the motion differences between marine organisms and marine garbage from the characteristics of the motion trajectory, such as the regularity of motion, the speed change pattern, and the frequency of change in the motion direction. For example, the motion of marine organisms often has certain biological behavior characteristics. They may swim and turn regularly to prey or avoid natural enemies, and the changes in their motion speed and direction are usually related to the instinctive behaviors of the organisms. In contrast, the motion of marine garbage under the action of external forces such as ocean currents and waves is relatively random and lacks the inherent regularity of biological motion. If the biological motion recognition model determines that the suspected marine garbage information is not a living being, then the intelligent detection system can determine that the suspected marine garbage information is the final marine garbage information. This judgment process effectively eliminates the interference of marine organisms on garbage detection, greatly improves the accuracy of determining the final marine garbage information, avoids misjudging living beings as garbage, ensures that the detection results can truly and accurately reflect the actual situation of marine garbage on the sea surface, and reduces incorrect operations in the subsequent processing process.

[0041] S105. Send the final marine garbage information and the garbage positioning information to the management terminal.

[0042] After the intelligent detection system determines the final marine garbage information and the garbage positioning information, it will send these key data to the management terminal in a timely and accurate manner. Before sending the information, the intelligent detection system will organize and package the final marine garbage information and the garbage positioning information, and arrange the detailed information such as the type, size, and quantity of the garbage, as well as its precise geographical location coordinates on the sea surface, in a specific data format to ensure the integrity and standardization of the data, so that the management terminal can receive and parse it accurately. Subsequently, the intelligent detection system establishes a connection with the management terminal through the communication module. The communication module can adopt various communication technologies, such as 4G and 5G wireless network communication technologies, which have high-speed and stable data transmission capabilities and can quickly transmit a large amount of data to the management terminal. In some special scenarios, if there is insufficient network signal coverage, satellite communication technology can also be used to ensure the reliability and timeliness of information transmission without being restricted by geographical location.

[0043] After the connection is successfully established, the intelligent detection system sends the packaged final marine garbage information and the garbage positioning information to the management terminal. After receiving the information, the management terminal can intuitively understand the specific situation and location distribution of the marine garbage on the sea surface. This enables the management personnel to quickly formulate targeted garbage cleaning plans based on this information.

[0044] In some embodiments, in the intelligent sea surface garbage detection system, the infrared thermal imager is a key device for obtaining the infrared thermal image information of the sea surface. Its stable operation is crucial for the reliability and continuity of the entire detection work. Therefore, the system will monitor the operating status of the infrared thermal imager in real time, and the power and signal strength are the key parameters for key monitoring. The system is configured with a dedicated monitoring module, which establishes a real-time communication connection with the infrared thermal imager and can continuously collect the operating data of the infrared thermal imager to monitor its operating status in real time. In terms of power monitoring, the monitoring module is connected to the power management system of the infrared thermal imager to obtain information such as the remaining battery power and charge and discharge status in real time. For the signal strength, the monitoring module interacts with the communication module of the infrared thermal imager to monitor the data transmission signal strength between it and the server or other devices in real time. Before the system runs, technicians will set reasonable parameter standard ranges for the power and signal strength according to the performance indicators and actual usage requirements of the infrared thermal imager. The monitoring module continuously compares the real-time collected power and signal strength parameters with the preset standard ranges. Once it is found that the power is lower than the lower threshold or the signal strength exceeds the normal range, the monitoring module will immediately trigger the maintenance reminder mechanism. Then, through the communication network of the system, the maintenance reminder information will be pushed to the management end. The management end can be the computer terminal of relevant staff, the mobile application, etc. After receiving the reminder, the staff can timely understand the abnormal situation of the infrared thermal imager and quickly arrange maintenance personnel to check, charge, repair the equipment or adjust the signal transmission equipment, etc.

[0045] In the embodiments of the present application, since multiple infrared thermal imagers are used to obtain the infrared thermal image information of the sea surface, the sea surface garbage recognition model is used to analyze the thermal image to determine the suspected garbage information and its location, the infrared thermal imager is used to track and obtain the movement trajectory and optimize it, and the biological movement recognition model and the ocean biological metabolic heat signal intensity change model are used to distinguish between organisms and garbage. Finally, the accurate information is sent to the management end. Therefore, it can comprehensively and accurately detect and locate the sea surface garbage in a complex environment, effectively solve the technical problems that the traditional detection technology is difficult to clearly detect the sea surface garbage under insufficient light and special meteorological conditions, unable to accurately track the movement trajectory of the garbage, and difficult to distinguish between marine organisms and garbage. Furthermore, it realizes the efficient and accurate detection of the sea surface garbage in a complex environment, provides reliable data support for marine garbage cleaning and marine environmental protection, and ensures the technical effects of marine ecological balance, navigation safety and the sustainable development of related industries.

[0046] After combining the above content, the following further describes the more specific process of the method provided in this embodiment. Please refer to Figure 2 , which is another process schematic diagram of the intelligent detection method for adaptive sea surface garbage in the embodiments of the present application.

[0047] S201. Combine the movement trajectory information and determine whether the suspected marine debris information corresponding to the movement trajectory information is a living organism through a biological movement recognition model, which is obtained by deep learning training in advance based on multiple movement trajectory samples of marine organisms and debris with annotations; After the intelligent detection system completes the acquisition of the suspected marine debris movement trajectory information, it will use the biological movement recognition model to further analyze it to determine whether the suspected marine debris is a living organism. This step has been described in detail in step S104 and will not be elaborated here.

[0048] S202. If it is impossible to determine whether the suspected marine debris information corresponding to the movement trajectory information is a living organism through the biological movement recognition model, obtain the thermal signal intensity data of the suspected marine debris at a set time; When the biological movement recognition model has difficulty determining whether the suspected marine debris is a marine organism or debris, the intelligent detection system activates a backup strategy to obtain the thermal signal intensity data of the suspected marine debris at a set time. Marine organisms generate unique thermal signals during their metabolic processes, and the thermal signal intensity changes with their physiological states. For example, when fish swim rapidly or engage in predation behavior, their metabolism accelerates and the thermal signal intensity increases; while when resting or in a relatively low-temperature environment, their metabolism slows down and the thermal signal intensity decreases. In contrast, marine debris basically does not exhibit changes in thermal signal intensity due to physiological activities.

[0049] The intelligent detection system can also be equipped with high-precision thermal signal sensors, which have extremely high sensitivity and stability and can accurately capture weak thermal signal intensity changes. The determination of the set time is not arbitrary but the result of considering many factors. Based on in-depth research on the thermal signal change laws of common marine organisms and the hardware performance of the system, after multiple experiments and optimizations, the intelligent detection system determines the set time to obtain the thermal signal intensity data every 5 minutes.

[0050] During the process of obtaining the thermal signal intensity data, the intelligent detection system performs real-time calibration and filtering on the data collected by the sensors. During the calibration process, the system regularly compares the data collected by the sensors with a standard thermal signal source and adjusts the parameters of the sensors through the comparison results to eliminate measurement errors caused by factors such as sensor aging and environmental temperature changes and ensure the accuracy of the data.

[0051] S203. Combine the thermal signal intensity data and determine the thermal signal similarity through a marine organism metabolic thermal signal intensity change model, which is constructed by deep learning in advance based on multiple thermal signal intensity change samples of marine organisms in different physiological states; After the intelligent detection system obtains the thermal signal intensity data of suspected marine garbage, it will use the marine biological metabolic thermal signal intensity change model to determine the thermal signal similarity. This model is based on deep learning technology and is constructed by learning the thermal signal intensity change samples of a large number of marine organisms in different physiological states. R & D personnel can collect in advance the thermal signal intensity change data of various marine organisms in different activity states and different environmental conditions. For example, the thermal signal intensity changes of different species of fish in cruising, predation, rest and other states are collected. These data cover a rich variety of biological species and diverse physiological states, providing a solid data foundation for the training of the model. During the model training process, deep learning algorithms are used to deeply mine and analyze these sample data, and it learns the change patterns, trends and change ranges of the thermal signal intensity of marine organisms under the influence of different physiological activities and environmental factors. For example, the algorithm will learn the rule that the thermal signal intensity of fish rises rapidly during predation and then gradually decreases during rest. Through the learning of a large number of samples, the model can accurately master the internal rules of the thermal signal intensity change of marine organisms.

[0052] The intelligent detection system inputs the obtained thermal signal intensity data of suspected marine garbage into this model. The model matches and analyzes the input thermal signal intensity data according to the learned thermal signal intensity change pattern of marine organisms. It calculates the similarity between the input data and various thermal signal intensity change patterns stored in the model. The calculation of similarity uses a complex algorithm, comprehensively considering multiple dimensions such as the change amplitude, change frequency, and duration of the thermal signal intensity. For example, if the change amplitude of the thermal signal intensity of the input data is similar to that of a certain marine organism in a specific physiological state, and the change frequency and duration also conform to the typical pattern of this organism, then the similarity is high; on the contrary, if the change characteristics of the data are quite different from the thermal signal intensity change pattern of known marine organisms, then the similarity is low. In this way, the model can accurately determine the similarity between the thermal signal intensity data of suspected marine garbage and the thermal signal intensity change pattern of marine organisms.

[0053] S204. If the thermal signal similarity is lower than the set biological feature threshold, then determine the suspected marine garbage information as the final marine garbage information.

[0054] After the intelligent detection system obtains the thermal signal similarity through the marine organism metabolic heat signal intensity change model, it compares it with the set biometric threshold, which is determined based on the statistical analysis of a large amount of marine organism thermal signal intensity change data and actual detection requirements. By analyzing various marine organism thermal signal intensity change samples, a reasonable threshold is determined to distinguish marine organisms from non-living things (i.e., marine debris). When the thermal signal similarity is lower than the set biometric threshold, the intelligent detection system has sufficient basis to determine that the suspected marine debris information is the final marine debris information, because a low thermal signal similarity means that the thermal signal intensity change pattern of the suspected marine debris is significantly different from the typical thermal signal change pattern of known marine organisms.

[0055] From the physiological characteristics of marine organisms, although the thermal signal intensity change ranges and patterns of different types of marine organisms vary, they all have certain rules and characteristics, while marine debris usually does not produce thermal signal intensity changes caused by physiological activities similar to those of marine organisms. For example, a floating plastic debris has a relatively stable thermal signal intensity and does not produce obvious and specific regular thermal signal intensity fluctuations like marine organisms due to metabolism and changes in activity state. When the thermal signal intensity data of the suspected marine debris has a very low similarity with the thermal signal pattern of marine organisms, it is very likely that it is marine debris.

[0056] In the embodiments of the present application, since multiple infrared thermal imagers are used to overcome the problem of cloudy day illumination to obtain the infrared thermal map of the sea surface, the marine debris recognition model is used to determine the suspected debris information and its location based on the thermal map, the infrared thermal imager is used to collect at time intervals and combined with various technologies to obtain the accurate movement trajectory, and the biological movement recognition model and the marine organism metabolic heat signal intensity change model are used to distinguish organisms from debris from the dual dimensions of movement trajectory and thermal signal. Finally, reliable information is sent to the management terminal, so it is possible to achieve the full-process efficient processing of marine debris from preliminary detection, accurate positioning, accurate identification to information transmission under complex special meteorological conditions such as thick fog or cloudy days. It effectively solves the problems of traditional detection technologies that it is difficult to clearly detect marine debris under insufficient light and special meteorological conditions, unable to accurately track the movement trajectory of debris, difficult to distinguish marine organisms from debris, and lag in information transmission, and thus realizes providing precise guidance for marine debris cleaning.

[0057] The intelligent detection system in the embodiments of the present invention application will be described from the perspective of hardware processing. Please refer to Figure 3 , which is a schematic structural diagram of an entity device of the intelligent detection system in the embodiments of the present application.

[0058] It should be noted that Figure 3 The structure of the intelligent detection system shown is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present invention.

[0059] As Figure 3 shown, the intelligent detection system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage section 308 into the random access memory (RAM) 303, such as executing the method described in the above embodiments. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0060] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a liquid crystal display (LCD), an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as required. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as required so that the computer program read from it can be installed into the storage section 308 as required.

[0061] Specifically, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are executed.

[0062] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0063] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings.

[0064] Specifically, the intelligent detection system of this embodiment includes a processor and a memory, and a computer program is stored on the memory. When the computer program is executed by the processor, the intelligent detection method for adaptive sea surface garbage provided in the above embodiment is implemented.

[0065] On the other hand, the present invention also provides a computer-readable storage medium, which may be included in the intelligent detection system described in the above embodiment; or it may exist separately without being assembled into the intelligent detection system. The above storage medium carries one or more computer programs. When the above one or more computer programs are executed by a processor of the intelligent detection system, the intelligent detection system implements the intelligent detection method for adaptive sea surface garbage provided in the above embodiment.

[0066] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present application.

[0067] As used in the foregoing embodiments, depending on the context, the term "when" may be interpreted to mean "if" or "after" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "when determining" or "if (the stated condition or event) is detected" may be interpreted to mean "if determining" or "in response to determining" or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)".

[0068] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented. The processes can be completed by relevant hardware instructed by a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the foregoing method embodiments. The foregoing storage medium includes various media that can store program codes, such as ROM or random access memory RAM, magnetic disks, or optical discs.

Claims

1. An adaptive intelligent detection method for sea surface garbage, applied to an intelligent detection system, characterized in that: The method comprises: Get the current sea surface light intensity information; If the light intensity information is less than a set light intensity threshold, obtaining sea surface infrared thermal image information through multiple infrared thermal imagers; Combined with the sea surface infrared thermal image information, suspected marine garbage information is determined through a marine garbage identification model, and garbage location information is obtained, wherein the marine garbage identification model is obtained in advance through deep learning training based on a plurality of marine garbage infrared thermal image samples with garbage annotations; Obtaining movement trajectory information of the suspected marine garbage information within a set time; Determining final marine garbage information in combination with the movement trajectory information; The final marine garbage information and the garbage location information are sent to a management terminal.

2. The method according to claim 1, characterized in that In combination with the sea surface infrared heat map information, the steps of determining suspected marine garbage information through a marine garbage identification model and obtaining garbage location information specifically include: Performing a preprocessing operation on the sea surface infrared thermal image information, wherein the preprocessing operation at least includes removing noise interference and enhancing image contrast; Smoothing random noise points in the sea surface infrared thermal image information by an image filtering algorithm; Using histogram equalization technology to stretch the image grayscale range in the sea surface infrared thermal map information; Using the marine garbage identification model to extract features from the sea surface infrared thermal image information to determine the shape, size and temperature distribution data of the thermal radiation emitted by the infrared thermal imager; Comparing the thermal radiation shape, size and temperature distribution data with the garbage feature library in the marine garbage identification model to determine similarity data; If the similarity data exceeds a set similarity threshold, it is determined to be suspected maritime spam.

3. The method according to claim 1, characterized in that: The step of obtaining the movement trajectory information of the suspected marine spam information within a set time specifically includes: Using multiple infrared thermal imagers to obtain infrared thermal image information of suspected garbage areas at set time intervals; Performing a preprocessing operation on the infrared thermal image information of the suspected garbage area, wherein the preprocessing operation at least includes noise reduction and histogram equalization; The suspected garbage in the sea surface infrared heat map information is tracked by using the optical flow method, and the location information of the suspected garbage at different times is combined to connect the position coordinates of each time point in sequence to determine the movement trajectory information; A Kalman filter algorithm is used to optimize the motion trajectory information.

4. The method according to claim 1, characterized in that: The step of determining the final marine garbage information in combination with the movement trajectory information specifically includes: In combination with the motion trajectory information, determining whether the suspected marine garbage information corresponding to the motion trajectory information is a living thing through a biological motion recognition model, wherein the biological motion recognition model is obtained in advance through deep learning training based on a plurality of labeled marine organisms and garbage motion trajectory samples; If it is not a living thing, the suspected marine spam information is determined to be the final marine spam information.

5. The method according to claim 4, characterized in that After the step of determining whether the suspected marine garbage information corresponding to the motion trajectory information is a living thing by using a biological motion recognition model in combination with the motion trajectory information, the method further includes: If the biological motion recognition model cannot determine whether the suspected marine garbage information corresponding to the motion trajectory information is a living thing, obtaining the thermal signal intensity data of the suspected marine garbage according to the set time; Combined with the heat signal intensity data, the heat signal similarity is determined by a marine organism metabolism heat signal intensity variation model, wherein the marine organism metabolism heat signal intensity variation model is constructed in advance by deep learning based on heat signal intensity variation samples of multiple marine organisms in different physiological states; If the thermal signal similarity is lower than a set biometric threshold, the suspected maritime spam is determined to be the final maritime spam.

6. The method according to claim 1, characterized in that After the step of obtaining sea surface infrared thermal image information through multiple infrared thermal imagers, it also includes: Analyzing the image quality information of the sea surface infrared thermal image information by an image analysis module, wherein the image information at least includes image edge clarity and contrast; If the image quality information decreases, and the decreasing trend is consistent with the feature of the lens being blocked, it is determined that the lens is blocked by seawater splashes or impurities; The micro jet device arranged around the lens is controlled to spray high-pressure cleaning gas toward the lens.

7. The method according to claim 1, characterized in that After the step of sending the final marine garbage information and the garbage location information to the management terminal, the method further includes: Performing real-time monitoring on the operating status of the infrared thermal imager, wherein the operating status at least includes the power level and signal strength of the device; When any parameter in the operating state exceeds the parameter standard range, maintenance reminder information is automatically generated and pushed to the management end.

8. A server, characterized in that: The server includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the server to execute the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a server, the server is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a server, the server is caused to execute the method according to any one of claims 1 to 7.

Images