A method and device for capturing sea surface garbage

Abstract

The embodiment of this specification discloses a method and device for capturing sea surface garbage, the method comprising: performing a depth-first search on the inspection nodes of a specified sea area to obtain the global coverage path of the device in the specified sea area for inspection; obtaining the inspection image taken by the preset camera and obtaining the posture data according to the preset six-axis IMU, performing image preprocessing on the inspection image and performing noise reduction processing on the posture data; wherein the image preprocessing includes: image enhancement processing and image denoising processing; inputting the processed inspection image and the posture data into the corresponding preset recognition model to determine the detection target; wherein the corresponding preset recognition model is determined after fuzzy image judgment based on the preprocessed inspection image. Based on the preset GPS locator, the real-time geographic location of the preset sea surface garbage capture device is obtained, and the local path planning is performed in combination with the location of the detection target to obtain the optimal local path, so as to realize the capture of the detection target based on the optimal local path.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more particularly to a method and device for capturing marine garbage. Background Art

[0002] Marine debris is currently defined as any persistent, manufactured or processed solid material discarded, disposed of or disposed of in the marine and coastal environment. With the continuous expansion of human living areas, from land to sea, due to improper daily activities of people, there is a large amount of garbage in the ocean; compared with land garbage, marine garbage is difficult to clean up, it has strong mobility and a wide distribution range, manual cleaning requires a lot of manpower and time, and marine garbage is a persistent environmental problem. It has a great impact on marine life and water quality, and these marine debris also pose a potential threat to human health. Therefore, the identification and capture of marine debris is a necessary technology to maintain environmental safety and health safety.

[0003] Due to the influence of strong winds and other factors in the marine environment, the existing marine identification and capture devices are easily affected by strong winds and waves when conducting autonomous patrols, resulting in the images taken by the sea surface garbage capture device being rotated, blurred, and other phenomena that are not conducive to identification, thereby increasing the processing work of capturing sea surface garbage, making it difficult to ensure the accuracy of capturing sea surface garbage. In addition, the existing capture of sea surface garbage generally identifies garbage based on artificially specified routes, which wastes the cost of human supervision and also makes the path cost and time cost of capturing sea surface garbage high. Summary of the invention

[0004] In order to solve the above technical problems, one or more embodiments of this specification provide a method and device for capturing sea surface garbage.

[0005] One or more embodiments of this specification adopt the following technical solutions:

[0006] On the one hand, one or more embodiments of this specification provide a method for capturing marine garbage, the method comprising:

[0007] Performing a depth-first search on the inspection nodes in the designated sea area, obtaining a global coverage path of the preset sea surface garbage capture device in the designated sea area, and controlling the preset sea surface garbage capture device to perform inspection based on the global coverage path;

[0008] Obtaining an inspection image taken by a preset camera on a preset sea surface garbage capture device, and obtaining the posture data of the sea surface garbage capture device based on a preset six-axis IMU on the preset sea surface garbage capture device, performing image preprocessing on the inspection image and performing noise reduction processing on the posture data; wherein the image preprocessing includes: image enhancement processing and image denoising processing;

[0009] Inputting the processed inspection image and the processed posture data into a corresponding preset recognition model to determine the detection target; wherein the corresponding preset recognition model is determined after performing fuzzy image determination based on the preprocessed inspection image;

[0010] Based on a GPS locator preset on the preset sea surface garbage capturing device, obtaining the real-time geographical location of the preset sea surface garbage capturing device;

[0011] Based on the real-time geographic location and the location of the detection target, local path planning is performed on the preset sea surface garbage capture device to obtain an optimal local path, so that the preset sea surface garbage capture device can capture the detection target based on the optimal local path.

[0012] Optionally, in one or more embodiments of the present specification, a depth-first search is performed on the inspection nodes of the designated sea area to obtain a global coverage path of the preset sea surface garbage capture device in the designated sea area, specifically including:

[0013] Obtaining the inspection nodes of the designated sea area, and randomly determining a starting inspection node among the inspection nodes for access marking;

[0014] Acquire an unvisited node adjacent to the start inspection node, so as to use the adjacent unvisited node as a next node, and acquire an unvisited node adjacent to the next node, so as to iteratively acquire the current node;

[0015] If it is determined that the current node does not have an adjacent unvisited node, then obtain the first search path where the current node is located, backtrack the current node, obtain the previous node corresponding to the current node, and iteratively obtain the second search path of the previous node; wherein each node in the second search path is a node that has not been visited during the iteration process;

[0016] Based on the access mark of each inspection node in the first search path and the second search path, the access time of each node is determined, and the inspection nodes are sorted based on the access time to obtain a global coverage path of the designated sea area.

[0017] Optionally, in one or more embodiments of the present specification, before inputting the processed inspection image and the processed posture data into a corresponding preset recognition model to determine the detection target, the method further includes:

[0018] Collect image samples of a preset recognition model and posture data samples of a preset sea surface garbage capture device corresponding to each image sample; wherein the image samples include: real target image samples and device visual samples;

[0019] Based on the device vision samples and the posture data samples as training samples, a preset generator network is trained and learned, and the generator network is iteratively trained by updating the loss function of the generator network to obtain a generator network that meets the requirements; wherein the loss function of the generator network is a mean square error;

[0020] Based on the real target image sample and the posture data sample, the preset discriminator network is trained and learned, and the discriminator network is iteratively trained to obtain a discriminator network that meets the requirements by updating the cross entropy loss function of the discriminator network;

[0021] Determining a preset image compensation model in the preset recognition model terminal according to the discriminator network that meets the requirements and the generator network that meets the requirements;

[0022] The real target image samples are learned based on a convolutional neural network model to obtain a preset foreign object recognition model, so as to determine whether to connect the preset image compensation model and the preset foreign object recognition model in series as a preset recognition model based on the processed inspection image; wherein the real target image samples are marked with the category of the detection target.

[0023] Optionally, in one or more embodiments of the present specification, after determining whether to connect the preset image compensation model and the preset foreign body recognition model in series as a preset recognition model based on the processed inspection image, the method further includes:

[0024] Exporting the deployable file corresponding to the preset recognition model to obtain an export file of the preset recognition model;

[0025] Performing a pruning operation on each level file of the preset recognition model in the export file to obtain a preset recognition model deployment file after pruning;

[0026] An edge device corresponding to the preset sea surface garbage capture device is determined to deploy the preset recognition model deployment file on the corresponding edge device.

[0027] Optionally, in one or more embodiments of the present specification, before performing local path planning on the preset sea surface garbage capture device to obtain an optimal local path based on the real-time geographical location and the location of the detection target, the method further includes:

[0028] Determine the position of the detection target in the inspection image and the boundary line of the inspection image, so as to determine the deviation value of the detection target from the center position of the inspection image based on the position in the inspection image and the boundary line of the inspection image;

[0029] The deviation value is transmitted to the preset sea surface garbage capturing device to adjust the posture data of the preset sea surface garbage capturing device to update the collection of the inspection image;

[0030] Acquire a first conversion relationship between a camera coordinate system and an image coordinate system corresponding to a camera on the current preset sea surface garbage capture device, and a second conversion relationship between the camera coordinate system and a world coordinate system;

[0031] The image position data of the detection target in the updated inspection image is obtained, and the image position data is converted based on the first conversion relationship and the second conversion relationship in sequence to obtain the position of the detection target.

[0032] Optionally, in one or more embodiments of the present specification, based on the real-time geographic location and the location of the detection target, local path planning is performed on the pre-placed sea surface garbage capture device to obtain an optimal local path, so that the pre-placed sea surface garbage capture device captures the detection target based on the optimal local path, specifically including:

[0033] Determining an initial local path of the preset sea surface garbage capture device according to the real-time geographical location and the location of the detection target;

[0034] Collecting environmental constraint data and dynamic characteristic data of the preset sea surface garbage capture device according to the preset sensor of the preset sea surface garbage capture device; wherein the environmental constraint data includes: sea current, wind speed; the dynamic characteristic data includes: minimum turning radius, acceleration threshold;

[0035] Determine the feasible path range of the preset sea surface garbage capture device at each time point based on the environmental constraint data and the dynamic characteristic data;

[0036] Based on the real-time environmental constraint data of the preset sea surface garbage capture device and the real-time dynamic characteristic data of the preset sea surface garbage capture device, the feasible path range is adjusted in real time, so as to search the feasible path range based on a preset optimization algorithm to obtain the best local path; wherein, the preset optimization algorithm includes: A algorithm, Dijkstra algorithm.

[0037] Optionally, in one or more embodiments of the present specification, after performing local path planning on the pre-placed sea surface garbage capture device to obtain an optimal local path based on the real-time geographical location and the location of the detection target, so that the pre-placed sea surface garbage capture device captures the detection target based on the optimal local path, the method further includes:

[0038] Acquire the current position of the preset sea surface garbage capture device, and determine the inspection node closest to the current position in the global coverage path based on the current position and the global coverage path of the designated sea area;

[0039] Determine the planned position of the nearest inspection node, and determine the deviation distance of the preset sea surface garbage capture device based on the current position and the planned position;

[0040] If it is determined that the deviation distance is greater than a preset threshold, the global coverage path is updated according to the current position.

[0041] On the other hand, one or more embodiments of the present specification provide a device, comprising: a main control board and a computing module, a navigation module, and a collection module connected to the main control board, so that the main control board controls each module to work; wherein,

[0042] The navigation module is used to perform a depth-first search on the inspection nodes in the designated sea area, obtain a global coverage path of the preset sea surface garbage capture device in the designated sea area, and control the preset sea surface garbage capture device to perform inspection based on the global coverage path;

[0043] The acquisition is used to obtain the inspection image taken by the preset camera on the preset sea surface garbage capture device, and obtain the posture data of the sea surface garbage capture device based on the preset six-axis IMU on the preset sea surface garbage capture device, perform image preprocessing on the inspection image and perform noise reduction on the posture data; wherein, the camera is partially located above the sea surface and partially located below the sea surface; the image preprocessing includes: image enhancement processing and image denoising processing;

[0044] The calculation module is used to input the processed inspection image and the processed posture data into a corresponding preset recognition model to determine the detection target; wherein the corresponding preset recognition model is determined after performing fuzzy image determination based on the preprocessed inspection image;

[0045] The navigation module is further used to obtain the real-time geographic location of the preset sea surface garbage capture device based on the GPS locator preset on the preset sea surface garbage capture device;

[0046] The calculation module is also used to perform local path planning for the preset sea surface garbage capture device based on the real-time geographic location and the position of the detection target to obtain an optimal local path, so that the preset sea surface garbage capture device can capture the detection target based on the optimal local path.

[0047] Optionally, in one or more embodiments of the present specification, the device further includes: a solar module, wherein the solar module includes: a solar controller, a solar photovoltaic panel connected to the solar controller, and a battery connected to the solar controller; the solar photovoltaic panel is controlled based on the solar controller to charge the battery, and the solar photovoltaic panel and the battery are controlled based on the power requirements of each module to output electrical energy to each module.

[0048] Optionally, in one or more embodiments of the present specification, the device further includes: a driving module consisting of a driver, a motor and a garbage capture mechanism; wherein the driver is used to receive a capture instruction generated by the main control board based on the calculation results of the calculation module, so as to drive the motor and the garbage capture mechanism to capture sea surface garbage based on the capture instruction.

[0049] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0050] Based on the inspection nodes of the designated sea area, a depth-first search is performed to obtain a global coverage path, and local path planning is performed after obtaining the real-time geographic location and the detection target location after real-time acquisition of the inspection image. The global coverage path is updated to obtain the best local path to capture sea surface garbage, and the process of autonomous patrol of the preset sea surface garbage capture device is realized, which improves the efficiency of inspection and garbage capture and reduces the cost of manual control inspection. In addition, the image compensation and image recognition method for the inspection image based on the series model of the preset image compensation model and the preset foreign object recognition model reduces the problem of inaccurate detection target recognition caused by rotation or blur of the captured image, and improves the ability of the preset sea surface garbage capture device to resist environmental interference. In addition, the solar modules installed on the device realize self-sustaining charging of each module, improve the endurance of the device, and facilitate the cleaning of marine garbage in a large range of sea areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0052] Figure 1 A schematic diagram of a method for capturing sea surface garbage provided in an embodiment of this specification;

[0053] Figure 2 A schematic diagram of a training process of a preset recognition model provided in an embodiment of this specification;

[0054] Figure 3 A schematic diagram of the structure of a device for capturing sea surface garbage provided in an embodiment of this specification;

[0055] Figure 4 A schematic diagram of the composition structure of a solar module provided in an embodiment of this specification;

[0056] Figure 5 A schematic diagram of battery power supply in a solar module provided in an embodiment of this specification. DETAILED DESCRIPTION

[0057] The embodiments of this specification provide a method and device for capturing marine garbage.

[0058] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0059] like Figure 1 As shown, the embodiment of this specification provides a schematic flow chart of a method for capturing sea surface garbage. Figure 2 It can be seen that in one or more embodiments of this specification, a method for capturing sea surface garbage includes the following steps:

[0060] S101: Perform a depth-first search on the inspection nodes in the designated sea area to obtain a global coverage path of a preset sea surface garbage capture device in the designated sea area, and control the preset sea surface garbage capture device to perform inspection based on the global coverage path.

[0061] In order to realize the autonomous patrol of the pre-set sea surface garbage capture device in the specified area, that is, the designated sea area, and complete the garbage capture. In the real-time of this specification, before executing the autonomous patrol process, a global path planning will be performed for the designated sea area. Specifically, in one or more embodiments of this specification, a depth-first search is performed on the inspection nodes of the designated sea area to obtain the global coverage path of the pre-set sea surface garbage capture device in the designated sea area, which specifically includes the following processes:

[0062] First, obtain the inspection nodes of the specified sea area, and randomly determine the starting inspection node in the inspection nodes for access marking. Then obtain the unvisited nodes adjacent to the starting inspection node, so as to use the adjacent unvisited nodes as the next nodes, and obtain the unvisited nodes adjacent to the next nodes, so as to iteratively obtain the current node. If it is determined that there are no adjacent unvisited nodes for the current node, then obtain the first search path where the current node is located, and backtrack the access nodes before the current node to obtain the previous node corresponding to the current node, so as to iteratively obtain the search path of the previous node as the second search path based on the previous node. It should be noted that each node in the second search path is a node that has not been visited during the iteration process. According to the access marks of each inspection node in the first search path and the second search path obtained in the above process, the access time of each node is determined, so that each inspection node is sorted according to the access time to obtain the global coverage path of the specified sea area.

[0063] S102: Obtain an inspection image taken by a preset camera on a preset sea surface garbage capture device, and obtain the posture data of the sea surface garbage capture device based on a preset six-axis IMU on the preset sea surface garbage capture device, perform image preprocessing on the inspection image and perform noise reduction on the posture data; wherein the image preprocessing includes: image enhancement processing and image denoising processing.

[0064] In order to enable the mass garbage capture device to identify and confirm the garbage and obstacles encountered during the autonomous patrol process so as to capture marine garbage in a timely manner, the inspection image taken by the preset camera on the preset sea surface garbage capture device is obtained in the embodiment of this specification. At the same time, in order to correct the blurred image caused by the influence of waves, the multi-point position information of the device is obtained by installing multiple six-axis IMUs in the embodiment of this specification, so as to correct the captured image and repair the blurred image. Then, in order to enhance the recognition accuracy, the inspection image is subjected to image enhancement and image denoising image preprocessing and the posture data is subjected to denoising.

[0065] S103: Inputting the processed inspection image and the processed posture data into a corresponding preset recognition model to determine the detection target; wherein the corresponding preset recognition model is determined after performing fuzzy image judgment based on the preprocessed inspection image.

[0066] After acquiring the processed inspection image and the processed posture data based on the above step S102, the acquired image and data are input into the corresponding preset recognition model, so as to determine the corresponding detection target based on the preset recognition model. It should be noted that the corresponding recognition model is determined after the preprocessed inspection image is judged as a blurred image. If the preprocessed inspection image is still determined to be a blurred image based on the empirical model, then the corresponding preset recognition model is a preset recognition model in which the preset image compensation model and the preset foreign body recognition model are connected in series. If it is determined that it is not a blurred image, then the preset foreign body recognition model is used as the preset recognition model.

[0067] Further, in one or more embodiments of the present specification, before inputting the processed inspection image and the processed posture data into the corresponding preset recognition model to determine the detection target, the method further includes the following process:

[0068] Collect image samples of the preset recognition model and the posture data samples of the preset sea surface garbage capture device corresponding to each image sample; wherein, the image samples include: real target image samples, device visual samples, and the posture sample data is based on the multiple six-axis IMUs installed on the preset sea surface garbage capture device, and the multi-point posture information is obtained. The acquisition of the posture data samples is helpful to correct the captured image and facilitates the repair of the blurred image in the subsequent image supplementation. Then, after obtaining the above sample data, in order to realize the compensation processing of the image, the embodiment of this specification uses the device visual samples and the posture data samples as training samples to train and learn the preset generator network, and iteratively trains the generator network by updating the loss function of the generator network to obtain a generator network that meets the requirements; wherein, it should be noted that the loss function of the generator network is the mean square error. Then, based on the real target image samples and the posture data samples, the preset discriminator network is trained and learned, and the cross entropy loss function of the discriminator network is updated to realize iterative training of the discriminator network and obtain a discriminator network that meets the requirements. According to the discriminator network and the generator network that meet the requirements obtained in the above process, the preset image compensation model in the preset recognition model terminal is formed. That is to say, the preset image compensation model in the embodiment of this specification is based on the generator network and the discriminator network. It can be understood that the generator network is responsible for converting the blurred image into a clear image, and its loss function adopts the mean square error; the discriminator network is used to distinguish the difference between the image generated by the generator and the real clear image, and its loss function is the cross entropy loss.

[0069] In addition, after compensating the image, in order to accurately identify the marine garbage present in the inspection image. In the embodiment of this specification, the real target image sample is learned based on the convolutional neural network model to obtain a preset foreign body recognition model, and the preset image compensation model is connected in series with the preset foreign body recognition model to obtain a preset recognition model. Among them, it should be noted that the real target image sample is marked with the category of the detection target. That is to say, in the embodiment of this specification, the category of garbage is marked for the inspection image compensated based on the preset image compensation model. It should be noted here that part of the camera in the preset sea surface garbage capture device is located above water and the other part is located underwater. Therefore, the inspection image collected by the camera device on the preset sea surface garbage capture device includes two groups of pictures, the above-water inspection image and the underwater inspection image. Based on this setting, when the subsequent marine garbage recognition is performed based on the above-water inspection image and the underwater inspection image, the problem of image blur and rotation caused by the influence of waves can be improved to improve the reliability of marine garbage recognition.

[0070] Specifically, in order to be able to use the preset recognition model obtained by training and learning in the above process, while reducing the size of the model and the amount of calculation during reasoning. In one or more embodiments of the present specification, after the preset image compensation model and the preset foreign body recognition model are connected in series to obtain the preset recognition model, the method also includes: exporting the deployable file corresponding to the preset recognition model to obtain the export file of the preset recognition model, and then pruning the files of each level of the preset recognition model in the export file to reduce the size of the model and the amount of calculation during reasoning, and obtaining the preset recognition model deployment file after pruning. Then determine the edge device corresponding to the preset sea surface garbage capture device to deploy the preset recognition model deployment file to the corresponding edge device.

[0071] S104: Based on a GPS locator preset on the preset sea surface garbage capturing device, a real-time geographical location of the preset sea surface garbage capturing device is obtained.

[0072] In order to obtain the geographical location of the device in real time, the real-time geographical location of the pre-set sea surface garbage capture device is obtained based on the GPS locator pre-set on the pre-set sea surface garbage capture device. It should be noted that the GPS locator is a global satellite positioning system, which is a medium-range circular orbit satellite navigation system. It can provide accurate positioning, speed measurement and high-precision time standards for most areas on the earth's surface, and can meet the needs of devices located anywhere in the world or in near-Earth space to continuously and accurately determine three-dimensional position, three-dimensional movement and time.

[0073] S105: Based on the real-time geographic location and the location of the detection target, local path planning is performed on the preset sea surface garbage capture device to obtain an optimal local path, so that the preset sea surface garbage capture device can capture the detection target based on the optimal local path.

[0074] In order to capture the detection target, that is, marine debris, in the embodiment of this specification, local path planning is performed based on the real-time geographic location obtained in the above step S104 and the position of the detection target obtained based on the inspection image to obtain the best local path. That is to say, in the process of autonomous inspection based on the global coverage path of the specified sea area, if marine debris is detected, the time elastic band algorithm will be used for local path planning to obtain the best local path to move to the position of the detection target, and then the PID algorithm of the preset sea surface garbage capture device is used to control the rotation of the motor to achieve the capture of marine debris.

[0075] Furthermore, in order to obtain the position of the detection target based on the inspection image, in one or more embodiments of the present specification, before performing local path planning on the preset sea surface garbage capture device to obtain the optimal local path based on the real-time geographic location and the position of the detection target, the method further includes the following steps:

[0076] First, the position of the detection target in the inspection image and the boundary line of the inspection image are determined, so as to determine the deviation value of the detection target from the center position of the inspection image according to the position in the inspection image and the boundary line of the inspection image. Then, the deviation value is transmitted to the preset sea surface garbage capture device, so as to adjust the posture data of the preset sea surface garbage capture device to update the acquisition of the inspection image. Then, the first conversion relationship between the camera coordinate system and the image coordinate system corresponding to the camera on the current preset sea surface garbage capture device, and the second conversion relationship between the camera coordinate system and the world coordinate system are obtained. The image position data of the detection target in the updated inspection image is obtained, so as to convert the image position data based on the first conversion relationship and the second conversion relationship in turn to obtain the position of the detection target.

[0077] Specifically, in one or more embodiments of the present specification, based on the real-time geographic location and the location of the detection target, local path planning is performed on the preset sea surface garbage capture device to obtain the optimal local path, so that the preset sea surface garbage capture device captures the detection target based on the optimal local path, specifically including the following processes:

[0078] First, the initial local path of the preset sea surface garbage capture device is determined according to the real-time geographic location and the location of the detection target. Then, the environmental constraint data and the dynamic characteristic data of the preset sea surface garbage capture device are collected according to the preset sensors of the preset sea surface garbage capture device. It should be noted that the environmental constraint data include: ocean current, wind speed, etc., and the dynamic characteristic data include: minimum turning radius, acceleration threshold, etc. According to the environmental constraint data and dynamic characteristic data obtained in the above process, the feasible path range of the preset sea surface garbage capture device at each time point is determined. Therefore, based on the real-time environmental constraint data of the preset sea surface garbage capture device and the real-time dynamic characteristic data of the preset sea surface garbage capture device, the feasible path range is adjusted in real time, so as to search the feasible path range based on the preset optimization algorithm to obtain the best local path. It should be noted that the preset optimization algorithm includes: A algorithm and Dijkstra algorithm.

[0079] In a certain embodiment of the present specification, the process of obtaining the best local path is: first obtain sensor data and perception information in the marine environment, such as ocean currents, wind speeds, obstacle positions, etc. This can be obtained by using sensor devices, meteorological data, hydrological data, etc. Then obtain the current position of the vessel where the preset sea surface capture device is located and the detection target position. Then, based on the dynamic characteristics and environmental constraints of the vessel where the preset sea surface capture device is located, a time elastic band is constructed. Among them, the time elastic band takes into account the minimum turning radius, acceleration limit, ocean current, wind speed and other factors of the vessel where the preset sea surface capture device is located, and defines the feasible path range of the vessel where the preset sea surface capture device is located at each time point. Then, according to real-time perception information and environmental changes, such as changes in ocean currents, wind speeds, and movement of obstacles, the time elastic band is dynamically updated. By taking these factors into consideration, the width and position of the band are updated. Search for the optimal path in the time elastic band. Optimization algorithms such as the A algorithm, the Dijkstra algorithm, or sampling-based methods such as the RRT and RRT algorithms can be used. To search for the best path. Path search should take into account dynamic factors such as ocean currents and wind speeds, and avoid obstacles. Path selection and smoothing: Select the best path from the searched paths and perform smoothing. Path selection can be based on a heuristic evaluation function or a cost function, and smoothing can be achieved through methods such as curve fitting and spline curves. Then, in order to enable the preset sea surface capture device to move based on the optimal local path, the optimal path is converted into actual control instructions for the vessel where the preset sea surface capture device is located, such as rudder angle, heading, speed, etc., so that the vessel where the preset sea surface capture device is located can execute according to the path. During the execution process, the heading and speed of the vessel need to be adjusted in real time to adapt to dynamic environmental changes.

[0080] Further, in order to realize the judgment that the preset sea surface garbage capture device deviates from the original path, in one or more embodiments of the present specification, based on the real-time geographical location and the position of the detection target, the local path planning of the preset sea surface garbage capture device is performed to obtain the best local path, so that the preset sea surface garbage capture device captures the detection target based on the best local path, and the method further includes the following process:

[0081] The current position of the preset sea surface garbage capture device is obtained, and based on the current position and the global coverage path of the designated sea area, the inspection node closest to the current position in the global coverage path is determined. Then the planned position of the nearest inspection node is determined, and the deviation distance of the preset sea surface garbage capture device is determined based on the current position and the planned position. If the deviation distance is greater than a preset threshold, the global coverage path corresponding to the preset sea surface garbage capture device is updated according to the current position.

[0082] like Figure 3 As shown, one or more embodiments of this specification provide a schematic diagram of the structure of a pre-placed sea surface garbage capture device. Figure 3 It can be seen that in one or more embodiments of this specification, a device for capturing marine garbage includes: a main control board and a computing module, a navigation module, and a camera module connected to the main control board, so that the main control board controls each module to work; wherein,

[0083] The navigation module is used to perform a depth-first search on the inspection nodes of the designated sea area, obtain a global coverage path of the preset sea surface garbage capture device in the designated sea area, and control the preset sea surface garbage capture device to perform inspection based on the global coverage path;

[0084] The camera module is used to obtain the inspection image taken by the preset camera on the preset sea surface garbage capture device, and pre-process the inspection image and input it into the preset recognition model to determine the detection target; wherein the preset recognition model is a series model of the preset image compensation model and the preset foreign body recognition model; the camera is partially located above the sea surface and partially located below the sea surface;

[0085] The navigation module is further used to obtain the real-time geographic location of the preset sea surface garbage capture device based on the GPS locator preset on the preset sea surface garbage capture device;

[0086] The calculation module is used to perform local path planning for the preset sea surface garbage capture device based on the real-time geographical location and the position of the detection target to obtain an optimal local path, so that the preset sea surface garbage capture device can capture the detection target based on the optimal local path.

[0087] Furthermore, the navigation module in the embodiments of this specification is composed of GPS and IMU, which, in addition to obtaining the geographical location of the device to facilitate path planning, is also used to obtain the posture information of the device to facilitate image correction.

[0088] Furthermore, in order to achieve the energy endurance of the pre-set sea surface garbage capture device, in one or more embodiments of the present specification, the device further comprises: a solar module, wherein Figure 4 The solar module shown includes: a solar controller, a solar photovoltaic panel connected to the solar controller, and a battery connected to the solar controller. The solar controller controls the solar photovoltaic panel to charge the battery, and controls the solar photovoltaic panel and the battery based on the power requirements of each module to output electrical energy to each module. That is to say, solar energy is used in the embodiments of this specification to improve the endurance of the pre-set sea surface garbage capture device. Solar panels are installed on the garbage capture device, and the electricity generated by the solar photovoltaic panel is used to stabilize the voltage and current through the solar controller to power the equipment and charge the battery, thereby improving the endurance of the device and completing garbage cleaning in a larger area. For example Figure 5 The schematic diagram of power generation of a battery in a solar module is shown. It can be seen that when the preset sea surface garbage capture device is working, the solar photovoltaic panel and the battery in the solar module will supply power to the working module, for example Figure 5 After the camera module in the middle acquires the inspection image, the acquired inspection image and the path information obtained by the GPS in the navigation module and the posture information obtained by the IMU are transmitted to the calculation module for calculation so that the driving module drives the capture mechanism on the preset sea surface garbage capture device to work. The battery and the solar photovoltaic panel will power the computing module, the driving module, the navigation module and the camera module, which are the modules involved in the work, to achieve the continuous endurance of the preset sea surface garbage capture device.

[0089] Further, in one or more embodiments of the present specification, in order to realize the capture action of the preset sea surface garbage capture device, the device in the embodiment of the present specification further includes: a driving module composed of a driver, a motor and a garbage capture mechanism. The driver is used to receive the capture instruction generated by the main control board based on the calculation result of the calculation module, so as to drive the motor and the garbage capture mechanism to capture the sea surface garbage based on the capture instruction.

[0090] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0091] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0092] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A method for capturing sea surface garbage, characterized in that: The method comprises: Performing a depth-first search on the inspection nodes in the designated sea area, obtaining a global coverage path of the preset sea surface garbage capture device in the designated sea area, and controlling the preset sea surface garbage capture device to perform inspection based on the global coverage path; Obtaining an inspection image taken by a preset camera on a preset sea surface garbage capture device, and obtaining the posture data of the sea surface garbage capture device based on a preset six-axis IMU on the preset sea surface garbage capture device, performing image preprocessing on the inspection image and performing noise reduction processing on the posture data; wherein the image preprocessing includes: image enhancement processing and image denoising processing; Input the processed inspection image and the processed posture data into the corresponding preset recognition model to determine the detection target; wherein the corresponding preset recognition model is determined after the preprocessed inspection image is subjected to blurred image judgment; if the preprocessed inspection image is determined to be still a blurred image based on the empirical model, then the corresponding preset recognition model is a preset recognition model obtained by serially connecting the preset image compensation model and the preset foreign body recognition model; if it is determined that it is not a blurred image, then the preset foreign body recognition model is used as the preset recognition model; Based on a GPS locator preset on the preset sea surface garbage capturing device, obtaining the real-time geographical location of the preset sea surface garbage capturing device; Based on the real-time geographical location and the location of the detection target, local path planning is performed on the preset sea surface garbage capture device to obtain an optimal local path, so that the preset sea surface garbage capture device captures the detection target based on the optimal local path; Before inputting the processed inspection image and the processed posture data into a corresponding preset recognition model to determine the detection target, the method further includes: Collect image samples of a preset recognition model and posture data samples of a preset sea surface garbage capture device corresponding to each image sample; wherein the image samples include: real target image samples and device visual samples; Based on the device vision samples and the posture data samples as training samples, a preset generator network is trained and learned, and the generator network is iteratively trained by updating the loss function of the generator network to obtain a generator network that meets the requirements; wherein the loss function of the generator network is a mean square error; Based on the real target image sample and the posture data sample, a preset discriminator network is trained and learned, and by updating the cross entropy loss function of the discriminator network, the discriminator network is iteratively trained to obtain a discriminator network that meets the requirements; Determining a preset image compensation model in the preset recognition model according to the discriminator network that meets the requirements and the generator network that meets the requirements; The real target image samples are learned based on a convolutional neural network model to obtain a preset foreign object recognition model; wherein the real target image samples are marked with the category of the detection target.

2. A method for capturing sea surface garbage according to claim 1, characterized in that: A depth-first search is performed on the inspection nodes of the designated sea area to obtain a global coverage path of the preset sea surface garbage capture device in the designated sea area, specifically including: Obtaining the inspection nodes of the designated sea area, and randomly determining a starting inspection node among the inspection nodes for access marking; Acquire an unvisited node adjacent to the start inspection node, so as to use the adjacent unvisited node as a next node, and acquire an unvisited node adjacent to the next node, so as to iteratively acquire the current node; If it is determined that the current node does not have an adjacent unvisited node, then obtain the first search path where the current node is located, backtrack the current node, obtain the previous node corresponding to the current node, and iteratively obtain the second search path of the previous node; wherein each node in the second search path is a node that has not been visited during the iteration process; Based on the access mark of each inspection node in the first search path and the second search path, the access time of each node is determined, and the inspection nodes are sorted based on the access time to obtain a global coverage path of the designated sea area.

3. A method for capturing sea surface garbage according to claim 1, characterized in that: After determining whether to connect the preset image compensation model and the preset foreign body recognition model in series as a preset recognition model based on the processed inspection image, the method further includes: Exporting the deployable file corresponding to the preset recognition model to obtain an export file of the preset recognition model; Performing a pruning operation on each level file of the preset recognition model in the export file to obtain a preset recognition model deployment file after pruning; An edge device corresponding to the preset sea surface garbage capture device is determined to deploy the preset recognition model deployment file on the corresponding edge device.

4. A method for capturing sea surface garbage according to claim 1, characterized in that: Before performing local path planning on the preset sea surface garbage capture device to obtain an optimal local path based on the real-time geographical location and the location of the detection target, the method further includes: Determine the position of the detection target in the inspection image and the boundary line of the inspection image, so as to determine the deviation value of the detection target from the center position of the inspection image based on the position in the inspection image and the boundary line of the inspection image; The deviation value is transmitted to the preset sea surface garbage capturing device to adjust the posture data of the preset sea surface garbage capturing device to update the collection of the inspection image; Acquire a first conversion relationship between a camera coordinate system and an image coordinate system corresponding to a camera on the current preset sea surface garbage capture device, and a second conversion relationship between the camera coordinate system and a world coordinate system; The image position data of the detection target in the updated inspection image is obtained, and the image position data is converted based on the first conversion relationship and the second conversion relationship in sequence to obtain the position of the detection target.

5. A method for capturing sea surface garbage according to claim 1, characterized in that: Based on the real-time geographical location and the position of the detection target, local path planning is performed on the preset sea surface garbage capture device to obtain an optimal local path, so that the preset sea surface garbage capture device captures the detection target based on the optimal local path, specifically including: Determining an initial local path of the preset sea surface garbage capture device according to the real-time geographical location and the location of the detection target; Collecting environmental constraint data and dynamic characteristic data of the preset sea surface garbage capture device according to the preset sensor of the preset sea surface garbage capture device; wherein the environmental constraint data includes: sea current, wind speed; the dynamic characteristic data includes: minimum turning radius, acceleration threshold; Determine the feasible path range of the preset sea surface garbage capture device at each time point based on the environmental constraint data and the dynamic characteristic data; Based on the real-time environmental constraint data of the preset sea surface garbage capture device and the real-time dynamic characteristic data of the preset sea surface garbage capture device, the feasible path range is adjusted in real time, so as to search the feasible path range based on a preset optimization algorithm to obtain the best local path; wherein, the preset optimization algorithm includes: A algorithm, Dijkstra algorithm.

6. A method for capturing sea surface garbage according to claim 1, characterized in that: Based on the real-time geographic location and the location of the detection target, after performing local path planning on the preset sea surface garbage capture device to obtain an optimal local path so that the preset sea surface garbage capture device captures the detection target based on the optimal local path, the method further includes: Acquire the current position of the preset sea surface garbage capture device, and determine the inspection node closest to the current position in the global coverage path based on the current position and the global coverage path of the designated sea area; Determine the planned position of the nearest inspection node, and determine the deviation distance of the preset sea surface garbage capture device based on the current position and the planned position; If it is determined that the deviation distance is greater than a preset threshold, the global coverage path is updated according to the current position.

7. A device for capturing sea surface garbage, characterized in that: The device comprises: a main control board and a computing module, a navigation module and a collection module connected to the main control board, so that the main control board controls each module to work; wherein, The navigation module is used to perform a depth-first search on the inspection nodes in the designated sea area, obtain a global coverage path of the preset sea surface garbage capture device in the designated sea area, and control the preset sea surface garbage capture device to perform inspection based on the global coverage path; The acquisition module is used to obtain the inspection image taken by the preset camera on the preset sea surface garbage capture device, and obtain the posture data of the sea surface garbage capture device based on the preset six-axis IMU on the preset sea surface garbage capture device, perform image preprocessing on the inspection image and perform noise reduction processing on the posture data; wherein, the camera is partially located above the sea surface and partially located below the sea surface; the image preprocessing includes: image enhancement processing and image denoising processing; The calculation module is used to input the processed inspection image and the processed posture data into the corresponding preset recognition model to determine the detection target; wherein the corresponding preset recognition model is determined after the preprocessed inspection image is subjected to blurred image judgment; if the preprocessed inspection image is determined to be still a blurred image based on the empirical model, then the corresponding preset recognition model is a preset recognition model in which the preset image compensation model and the preset foreign body recognition model are connected in series; if it is determined that it is not a blurred image, then the preset foreign body recognition model is used as the preset recognition model; The navigation module is further used to obtain the real-time geographic location of the preset sea surface garbage capture device based on the GPS locator preset on the preset sea surface garbage capture device; The calculation module is further used to perform local path planning on the preset sea surface garbage capture device to obtain an optimal local path based on the real-time geographical location and the position of the detection target, so that the preset sea surface garbage capture device captures the detection target based on the optimal local path; Before inputting the processed inspection image and the processed posture data into the corresponding preset recognition model to determine the detection target, the method further includes: Collect image samples of a preset recognition model and posture data samples of a preset sea surface garbage capture device corresponding to each image sample; wherein the image samples include: real target image samples and device visual samples; Based on the device vision samples and the posture data samples as training samples, a preset generator network is trained and learned, and the generator network is iteratively trained by updating the loss function of the generator network to obtain a generator network that meets the requirements; wherein the loss function of the generator network is a mean square error; Based on the real target image sample and the posture data sample, a preset discriminator network is trained and learned, and by updating the cross entropy loss function of the discriminator network, the discriminator network is iteratively trained to obtain a discriminator network that meets the requirements; Determining a preset image compensation model in the preset recognition model according to the discriminator network that meets the requirements and the generator network that meets the requirements; The real target image samples are learned based on a convolutional neural network model to obtain a preset foreign object recognition model; wherein the real target image samples are marked with the category of the detection target.

8. A device for capturing marine garbage according to claim 7, characterized in that: The device also includes: a solar module, wherein the solar module includes: a solar controller, a solar photovoltaic panel connected to the solar controller, and a battery connected to the solar controller; the solar photovoltaic panel is controlled based on the solar controller to charge the battery, and the solar photovoltaic panel and the battery are controlled based on the power demand of each module to output electrical energy to each module.

9. A device for capturing marine garbage according to claim 8, characterized in that: The device also includes: a driving module consisting of a driver, a motor and a garbage capturing mechanism; wherein the driver is used to receive a capturing instruction generated by the main control board based on the calculation results of the computing module, so as to drive the motor and the garbage capturing mechanism to capture sea surface garbage based on the capturing instruction.

Images