Underwater Environment Visual Perception System for the Cleaning Process of Underwater Robots

Abstract

The present invention relates to the field of image recognition technology, and particularly to an underwater environment visual perception system for the cleaning process of an underwater robot. First, based on the characteristic that the dirt area has a relatively high tightness compared to other noise areas, the texture tightness is determined according to the uniformity of the gray distribution and the complexity of the gray texture in each local area of each area to be recognized and segmented. Further, according to the characteristic that the dirt area usually remains fixed compared to interference factors such as dynamically changing water bubbles, attached seaweeds, and dynamically changing type noises, the degree of regional swing representing the possibility of regional interference is determined according to the inter-frame position change situation of each area to be recognized and segmented in the underwater environment gray image. Further, the possibility of the noise area is comprehensively characterized by the texture tightness and the degree of regional swing, so as to screen out more accurate real dirt areas according to the possibility of the noise area, making the accuracy of underwater environment dirt recognition higher.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to an underwater environment visual perception system for the cleaning process of an underwater robot. Background Art

[0002] The core task of the underwater environment visual perception system in the cleaning work of an underwater robot is to sense and analyze the underwater environment through sensors such as cameras to achieve efficient and intelligent cleaning. Among them, an underwater robot refers to an automated device that can perform tasks in an underwater environment, also known as an underwater drone. In the cleaning operation, an autonomous underwater vehicle (AUV) with autonomous navigation capabilities is usually adopted. It relies on a preset program or sensors to sense the environment and complete tasks independently; and it is usually equipped with thrusters, cameras, environmental sensors, and cleaning tools such as brushes and high-pressure water guns. The goal of underwater cleaning is usually to remove dirt on the surface of the hull, marine facilities, storage tanks, or underwater structures. In order to determine which areas need to be cleaned, it is first necessary to identify the positions corresponding to the dirt, so as to plan the cleaning route through the AUV according to the identified dirt positions.

[0003] The prior art usually adopts the method of image recognition, and uses the semantic segmentation method to segment the required dirt area after determining the underwater environment image; however, due to the influence of turbid water quality and high depth, the underwater light intensity is usually low, making the clarity of the collected underwater environment image usually poor. At this time, the obtained underwater environment image is greatly affected by other factors such as noise. If the semantic segmentation is directly performed on the underwater environment image, bubbles, attached seaweeds, and dynamic change type noises will affect the recognition of the dirt area, making the dirt area segmented by directly using the semantic segmentation method inaccurate and reducing the accuracy of underwater environment dirt recognition. Summary of the Invention

[0004] The present application provides an underwater environment visual perception system for the cleaning process of an underwater robot. First, based on the characteristic that the dirt area has a relatively high compactness compared to other noise areas, the texture compactness is determined according to the uniformity of the gray-scale distribution and the complexity of the gray-scale texture in each local area of each region to be recognized and segmented. Further, based on the characteristic that the dirt area usually remains fixed compared to interfering factors such as dynamically changing water bubbles, attached seaweeds, and dynamically changing noise-like factors, the degree of regional swing representing the possibility of regional interference is determined according to the inter-frame position change of each region to be recognized and segmented in the underwater environment gray-scale image. Further, the possibility of the noise area is comprehensively characterized based on the texture compactness and the degree of regional swing, so as to screen out more accurate real dirt areas according to the possibility of the noise area, solving the problem that the dirt areas segmented by directly using the semantic segmentation method are not accurate enough, and making the accuracy of underwater environment dirt recognition higher.

[0005] The present application provides an underwater environment visual perception system for the cleaning process of an underwater robot, including:

[0006] A data acquisition and preprocessing module, configured to obtain each frame of underwater environment gray-scale image during the cleaning process of the underwater robot in chronological order; perform semantic segmentation on the underwater environment gray-scale image to obtain corresponding regions to be recognized and segmented;

[0007] A first determination module, configured to determine the texture compactness of each region to be recognized and segmented according to the uniformity of the gray-scale distribution and the complexity of the gray-scale texture in each local area of each region to be recognized and segmented in each frame of underwater environment gray-scale image;

[0008] A second determination module, configured to determine the degree of regional swing of each region to be recognized and segmented according to the inter-frame position change of each region to be recognized and segmented in the underwater environment gray-scale image;

[0009] An underwater environment dirt recognition module, configured to determine the possibility of the noise area of each region to be recognized and segmented in the underwater environment gray-scale image according to the texture compactness and the degree of regional swing; screen out the real dirt areas in the underwater environment gray-scale image according to the possibility of the noise area; perform underwater environment dirt recognition according to the real dirt areas.

[0010] Further, the process of obtaining the regions to be recognized and segmented includes:

[0011] Input the underwater environment gray-scale image into a trained U-Net convolutional neural network, and output each region to be recognized and segmented in the underwater environment gray-scale image; wherein, the loss function uses the cross-entropy loss function.

[0012] Further, the process of obtaining the texture tightness includes:

[0013] In the underwater environment grayscale image, each region to be recognized and segmented is divided into at least two local regions with the same shape and area; the gray-level co-occurrence matrix of each local region is constructed by the gray-level co-occurrence matrix algorithm and the corresponding energy value is calculated; the average value of the energy values of the gray-level co-occurrence matrices of all local regions in each region to be recognized and segmented is used as the reference energy value of each region to be recognized and segmented;

[0014] In each region to be recognized and segmented, the energy divergence of each region to be recognized and segmented is determined according to the disorder degree of the distribution of the energy values of the gray-level co-occurrence matrices of each local region;

[0015] According to the energy divergence and the reference energy value, the texture tightness of each region to be recognized and segmented is determined; there is a negative correlation between the energy divergence and the texture tightness, and there is a positive correlation between the reference energy value and the texture tightness.

[0016] Further, the process of obtaining the energy divergence includes:

[0017] The variance of the energy values of the gray-level co-occurrence matrices of all local regions in each region to be recognized and segmented is used as the energy divergence.

[0018] Further, the process of determining the texture tightness of each region to be recognized and segmented according to the energy divergence and the reference energy value includes:

[0019] The texture tightness of each region to be recognized and segmented is determined according to the product between the negative correlation mapping value of the energy divergence and the reference energy value.

[0020] Further, the process of obtaining the degree of regional swing includes:

[0021] The centroid coordinate points of each region to be recognized and segmented in each frame of the underwater environment grayscale image are counted; each other underwater environment grayscale image outside each frame of the underwater environment grayscale image is used as the reference grayscale image of each frame of the underwater environment grayscale image;

[0022] For any frame of the underwater environment grayscale image: each region to be recognized and segmented in the underwater environment grayscale image is sequentially used as the target feature region; the centroid coordinate point of the target feature region is used as the target coordinate point; in each reference grayscale image corresponding to the underwater environment grayscale image, the region to be recognized and segmented corresponding to the centroid coordinate point with the smallest distance from the target coordinate point is used as the matching feature region of the target feature region in each reference grayscale image;

[0023] The Euclidean distance between the centroid coordinate point of the target feature region and the centroid coordinate points of the matching feature regions in each corresponding reference grayscale image is used as the reference change distance corresponding to the target feature region in each reference grayscale image; the mean value of the reference change distances corresponding to the target feature region in all reference grayscale images is subjected to a positive correlation mapping to determine the degree of regional swing of the target feature region.

[0024] Further, the process of obtaining the possibility of the noise region includes:

[0025] Normalize the product between the negative correlation mapping value of the texture compactness and the degree of regional swing to determine the possibility of the noise region for each region to be recognized and segmented in the underwater environment grayscale image.

[0026] Further, the process of obtaining the true dirt region includes:

[0027] Regard the regions to be recognized and segmented with the possibility of the noise region less than the preset noise threshold as the true dirt regions.

[0028] Further, the preset noise threshold is set to 0.53.

[0029] Further, the process of dividing each region to be recognized and segmented into at least two local regions with the same shape and area includes:

[0030] According to the division specification of 5×5, divide each region to be recognized and segmented into 25 local regions with the same size and in the shape of rectangles.

[0031] This application has the following beneficial effects:

[0032] First of all, the key point to be processed in this application is to process each region to be recognized and segmented selected according to the difference features between the dirt region and the noise region, so as to screen out the required true dirt regions according to the obtained possibility of the noise region; therefore, it is necessary to analyze the difference features between the dirt region and the noise region.

[0033] For the dirt attached to the surface of underwater facilities, its main visual characteristics are dense arrangement, overall coverage and compact accumulation; while the main visual characteristics of the noise regions corresponding to water bubbles, attached seaweeds and dynamic change noises are dispersion and sparsity, such as floating algae or attached algae, air bubbles, etc. Therefore, the dirt region has better compactness than the noise region, that is, the primary distinguishing means is to measure the compactness of each region to be recognized and segmented, so as to indirectly characterize the possibility of belonging to the noise region; therefore, in this application, the texture compactness is determined according to the uniformity of the gray distribution and the complexity of the gray texture of each local region in each region to be recognized and segmented, so that when the texture compactness is greater, the corresponding region to be recognized and segmented is more likely to belong to the dirt region, that is, the possibility of belonging to the noise region is smaller.

[0034] Furthermore, it is necessary to consider that accidental situations may cause the visual characteristics presented by some floating algae to also show dense arrangement. Therefore, in order to reduce the influence of accidental situations on the recognition of the dirt region, it is necessary to analyze in combination with other distinguishing features on the basis of the texture compactness. For the dirt attached to the surface of underwater facilities, the dirt is usually attached to the surface of the underwater facilities and does not change its position over time; while under the influence of water flow, algae or bubble-like noises will show dynamic change characteristics. Specifically, the algae will show a swaying characteristic under the influence of water flow, and the air bubbles will generate a certain displacement under the influence of water flow; therefore, the noise region will have a significant position change in the time sequence compared with the dirt region. Therefore, in this application, the degree of regional swing is determined according to the inter-frame position change of each region to be recognized and segmented in the underwater environment gray image, so that when the degree of regional swing is greater, the corresponding region to be recognized and segmented is more likely to belong to the noise region, that is, the possibility of belonging to the dirt region is smaller.

[0035] Finally, by comprehensively considering the dirt-noise distinguishing features characterized by the texture compactness and the degree of regional swing, the possibility of the noise region of each recognition feature region is determined, so as to screen out the more accurate real dirt region in the underwater environment gray image according to the possibility of the noise region, making the accuracy of underwater environment dirt recognition higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings.

[0037] Figure 1 It is a structural block diagram of an underwater environment visual perception system for an underwater robot cleaning operation process provided by an embodiment of the present invention. Detailed Implementation Modes

[0038] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation modes, structures, features, and effects of an underwater environment visual perception system for an underwater robot during the cleaning process proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment, and specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as implying or suggesting relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0040] The following specifically describes in conjunction with the accompanying drawings the specific solution of an underwater environment visual perception system for an underwater robot during the cleaning process provided by the present invention.

[0041] An embodiment of the present application provides an underwater environment visual perception system for an underwater robot during the cleaning process. Please refer to Figure 1 which shows a structural block diagram of an underwater environment visual perception system for an underwater robot during the cleaning process provided by an embodiment of the present invention. The system includes a data acquisition and preprocessing module 101, a first determination module 102, a second determination module 103, and an underwater environment dirt identification module 104, where:

[0042] The data acquisition and preprocessing module 101 is configured to, in chronological order, acquire each frame of underwater environment grayscale image during the cleaning process of the underwater robot; perform semantic segmentation on the underwater environment grayscale image to obtain corresponding regions to be identified and segmented.

[0043] In a specific implementation manner of the embodiment of the present invention, an AUV robot is selected as the underwater robot. A high-definition underwater camera is installed at the front end of the AUV robot and is directed at the surface of the underwater facility in the underwater environment to capture images, obtaining the initial underwater environment images for each frame; and in order to facilitate subsequent analysis of the grayscale, the initial underwater environment images are further grayscaled to obtain the grayscale underwater environment images for each frame to be analyzed. Among them, the waterproof level of the high-definition underwater camera is set to IP68, and a high-definition underwater camera with a higher waterproof level can be used according to the specific implementation environment; the resolution of the initial underwater environment images is 3840×2160, and the frame rate for capturing the initial underwater environment images is set to 30fps. The resolution size and frame rate size can both be adjusted according to the specific implementation environment. It should be noted that the total number of grayscale underwater environment images analyzed in the embodiment of the present invention is set to 60, and all the initial underwater environment images analyzed are images of the same underwater facility surface captured by the high-definition underwater camera at different positions, so as to ensure the accuracy of the subsequent analysis of the regional swing degree, and no further elaboration will be made here.

[0044] After obtaining the grayscale underwater environment images, first, the semantic segmentation method is used to determine each region to be recognized and segmented that needs to be analyzed in the embodiment of the present invention; the region to be recognized and segmented may be the required dirt region, or may be the corresponding noise regions such as water bubbles, attached seaweeds, and dynamically changing noises, etc. Therefore, it is necessary to further analyze the region to be recognized and segmented to screen out the required real dirt region.

[0045] Preferably, in some possible implementation manners of the embodiment of the present invention, the process of obtaining the region to be recognized and segmented includes:

[0046] The grayscale underwater environment images are input into the trained U-Net convolutional neural network, and each region to be recognized and segmented in the grayscale underwater environment images is output; among them, the loss function uses the cross-entropy loss function. It should be noted that the U-Net convolutional neural network is a well-known semantic segmentation technical means in the art, and the implementer can adopt other methods according to the specific implementation environment, and no further limitation and elaboration will be made here.

[0047] The first determination module 102 is used to determine the texture tightness of each region to be recognized and segmented according to the uniformity of the grayscale distribution and the complexity of the grayscale texture of each local region in each frame of the grayscale underwater environment image.

[0048] First of all, the key point to be processed in this application is to process each region to be recognized and segmented according to the difference features between the dirt region and the noise region, so as to screen out the required real dirt region according to the obtained possibility of the noise region; therefore, it is necessary to analyze the difference features between the dirt region and the noise region.

[0049] For the dirt attached to the surface of underwater facilities, its main visual characteristics are dense arrangement, overall coverage, and compact accumulation; while the main visual characteristics of the noise regions corresponding to water bubbles, attached seaweeds, and dynamically changing noises are dispersion and sparsity, such as floating algae or attached algae, bubbles, etc. Therefore, the dirt region has better compactness than the noise region, that is, the primary means of distinction is to measure the compactness of each region to be recognized and segmented, so as to indirectly characterize the possibility of belonging to the noise region; therefore, in this application, the texture compactness is determined according to the uniformity of the gray distribution and the complexity of the gray texture in each local region of each region to be recognized and segmented, so that the greater the texture compactness, the more likely the corresponding region to be recognized and segmented belongs to the dirt region, that is, the smaller the possibility of belonging to the noise region.

[0050] Preferably, in some possible implementation manners of the embodiments of the present invention, the process of obtaining the texture compactness includes:

[0051] In the grayscale image of the underwater environment, each region to be recognized and segmented is divided into at least two local regions with the same shape and area; the gray-level co-occurrence matrix of each local region is constructed by the gray-level co-occurrence matrix algorithm and the corresponding energy value is calculated; the average value of the energy values of the gray-level co-occurrence matrices of all local regions in each region to be recognized and segmented is used as the reference energy value of each region to be recognized and segmented. For the gray-level co-occurrence matrix, the higher the energy value, the more uniform or consistent the texture of the corresponding image region; therefore, the larger the average value of the energy values of the gray-level co-occurrence matrices of all local regions in the region to be recognized and segmented, that is, the larger the reference energy value, the more uniform the local gray texture distribution of the region to be recognized and segmented in the dimension of local analysis, that is, the more in line with the characteristics of the dense texture distribution of the dirt region. It should be noted that the gray-level co-occurrence matrix is a well-known technical means in the art and will not be further defined and described here.

[0052] In a specific implementation manner of the embodiments of the present invention, the process of dividing each region to be recognized and segmented into at least two local regions with the same shape and area includes: dividing each region to be recognized and segmented into 25 local regions with the same size and rectangular shape according to the 5×5 division specification. It should be noted that the implementer can adopt other methods or specifications for dividing the region to be recognized and segmented according to the specific implementation environment, which will not be further described here.

[0053] In each segmentation region to be recognized, according to the chaotic situation of the energy value distribution of the gray-level co-occurrence matrix of each local region, the energy divergence of each segmentation region to be recognized is determined. Since the energy value characterizes the texture uniformity of each local region, if the overall distribution of the segmentation region to be recognized is relatively compact, the texture uniformity of each local region should be relatively concentrated and overall relatively uniform. Therefore, when the degree of energy value divergence obtained from the chaotic situation of the energy value distribution is greater, it indicates that the texture uniformity distribution of each local region is more discrete, and the segmentation region to be recognized does not conform to the characteristic of the relatively compact texture distribution of the dirt region more.

[0054] Preferably, in some possible implementation manners of the embodiments of the present invention, the process of obtaining the energy divergence includes:

[0055] The variance of the energy values of the gray-level co-occurrence matrices of all local regions in each segmentation region to be recognized is used as the energy divergence. For any set of data, the greater the variance, the more discrete the corresponding data. Therefore, the energy divergence is determined according to the variance of the energy values of the gray-level co-occurrence matrices of all local regions, so that when the energy divergence is greater, the energy value distribution of the gray-level co-occurrence matrices of all corresponding local regions is more discrete, that is, the corresponding segmentation region to be recognized does not conform to the characteristic of the relatively compact texture distribution of the dirt region more.

[0056] Further, the energy divergence and the reference energy value are combined, and according to the energy divergence and the reference energy value, the texture compactness of each segmentation region to be recognized is determined; since the smaller the energy divergence and the greater the reference energy value, the more the corresponding segmentation region to be recognized conforms to the characteristic of the relatively compact texture distribution of the dirt region, that is, the greater the texture compactness, the more likely the corresponding segmentation region to be recognized is the dirt region, so the energy divergence and the texture compactness are negatively correlated, and the reference energy value and the texture compactness are positively correlated.

[0057] Preferably, in some possible implementation manners of the embodiments of the present invention, the process of determining the texture compactness of each segmentation region to be recognized according to the energy divergence and the reference energy value includes:

[0058] The texture compactness of each segmentation region to be recognized is determined according to the product between the negative correlation mapping value of the energy divergence and the reference energy value. It should be noted that in addition to the product, the implementer can also calculate the texture compactness by other methods according to the relevant relationship, such as the normalized values of the mean value and the sum value, etc., which will not be further elaborated here.

[0059] In a specific implementation manner of the embodiments of the present invention, the process of obtaining the texture compactness is expressed by the formula: ; where is the The texture compactness of the th region to be recognized and segmented in the frame underwater environment grayscale image; is the variance of the energy values of the gray-level co-occurrence matrices of all local regions in the th region to be recognized and segmented in the th frame of the underwater environment grayscale image, that is, the energy divergence; is the number of local regions in the th region to be recognized and segmented in the th frame of the underwater environment grayscale image; is the energy value of the gray-level co-occurrence matrix of the th local region in the th region to be recognized and segmented in the th frame of the underwater environment grayscale image; is the average value of the energy values of the gray-level co-occurrence matrices of all local regions in the th region to be recognized and segmented in the th frame of the underwater environment grayscale image, that is, the reference energy value; is the exponential function with the natural constant as the base.

[0060] The second determination module 103 is configured to determine the regional swing degree of each region to be recognized and segmented according to the inter-frame position change situation of each region to be recognized and segmented in the underwater environment grayscale image.

[0061] Furthermore, it is necessary to consider that there may be accidental situations where the visual features presented by some floating algae also show a tight arrangement. Therefore, in order to reduce the influence of accidental situations on the recognition of the dirt area, it is necessary to analyze in combination with other distinguishing features on the basis of texture compactness. For the dirt attached to the surface of the underwater facility, the dirt is usually attached to the surface of the underwater facility and does not change its position over time; under the influence of water flow, algae or bubble-like noises will show dynamic change characteristics. Specifically, algae will show a swaying characteristic under the influence of water flow, and bubbles will produce a certain displacement under the influence of water flow; therefore, the noise area will have a significant position change in the time sequence compared with the dirt area. Therefore, in this application, the regional swing degree is determined according to the inter-frame position change situation of each region to be recognized and segmented in the underwater environment grayscale image, so that the greater the regional swing degree, the more likely the corresponding region to be recognized and segmented belongs to the noise area, that is, the less likely it belongs to the dirt area.

[0062] Preferably, in some possible implementation manners of the embodiments of the present invention, the process of obtaining the regional swing degree includes:

[0063] Statistically calculate the centroid coordinate points of each region to be recognized and segmented in each frame of the underwater environment grayscale image; use other underwater environment grayscale images outside each frame of the underwater environment grayscale image as the reference grayscale images for each frame of the underwater environment grayscale image. Among them, the centroid coordinate points are used to measure the position of the region to be recognized and segmented. Other coordinate points such as the center-of-mass coordinate points can also be selected for analysis, and no further elaboration will be made here.

[0064] For any frame of the underwater environment grayscale image: sequentially take each region to be recognized and segmented in the underwater environment grayscale image as the target feature region; take the centroid coordinate point of the target feature region as the target coordinate point; in each reference grayscale image corresponding to the underwater environment grayscale image, take the region to be recognized and segmented corresponding to the centroid coordinate point with the smallest distance from the target coordinate point as the matching feature region of the target feature region in each reference grayscale image.

[0065] For the target feature region, if it belongs to the region corresponding to algae or bubble-like noise, due to the corresponding dynamic change characteristics, the target feature region will have a certain position change in different frames of the underwater environment grayscale image; while the real dirt will not be affected by the water flow, so that the target feature region corresponding to the real dirt usually will not change its position or change less in different frames of the underwater environment grayscale image; therefore, for the target feature region, the smaller the overall distance between its corresponding target coordinate point and the centroid coordinate points of each matching feature region, the smaller the position change of the target feature region in different frames of the underwater environment grayscale image, and the more it conforms to the characteristic of the unchanged position of the dirt region. Therefore, further take the Euclidean distance between the centroid coordinate point of the target feature region and the centroid coordinate points of the matching feature regions in each corresponding reference grayscale image as the reference change distance corresponding to the target feature region in each reference grayscale image; perform a positive correlation mapping on the mean value of the reference change distances corresponding to the target feature region in all reference grayscale images to determine the regional swing degree of the target feature region. The smaller the mean value of the reference change distances corresponding in all reference grayscale images, that is, the smaller the regional swing degree, the more likely the corresponding target feature region belongs to the region corresponding to the dirt, that is, the less likely it belongs to the region affected by noise.

[0066] In a specific implementation manner of the embodiment of the present invention, the process of obtaining the regional swing degree is expressed by the formula: ; where is the regional swing degree of the th region to be recognized and segmented in the th frame of the underwater environment grayscale image; is the total number of underwater environment grayscale images; is the th frame of the underwater environment grayscale image, and The Euclidean distance between the centroid coordinate point of a to-be-recognized segmented area and the centroid coordinate point of the corresponding matching feature area in the reference grayscale image, that is, the reference change distance; is a preset positive correlation mapping parameter, which is set to 0.1 in the embodiments of the present invention, and is used to prevent the calculated area swing degree from being 0, reduce the influence on the subsequent calculation of the possibility of noise areas, and the size of the preset positive correlation mapping parameter can be adjusted according to the specific implementation environment; is a linear normalization function, and the implementer can adjust the normalization method according to the specific implementation environment.

[0067] The underwater environment dirt recognition module 104 is configured to determine the possibility of a noise area for each to-be-recognized segmented area in the underwater environment grayscale image according to the texture tightness and the area swing degree; screen out the real dirt areas in the underwater environment grayscale image according to the possibility of the noise area; perform underwater environment dirt recognition according to the real dirt areas.

[0068] Finally, comprehensively considering the dirt noise discrimination features characterized by the texture tightness and the area swing degree, determine the possibility of the noise area for each recognized feature area, so as to screen out more accurate real dirt areas in the underwater environment grayscale image according to the possibility of the noise area, making the accuracy of the underwater environment dirt recognition higher.

[0069] Preferably, in some possible implementation manners of the embodiments of the present invention, the process of obtaining the possibility of the noise area includes:

[0070] Since the greater the texture tightness and the smaller the area swing degree, the more likely the corresponding to-be-recognized segmented area is a dirt area, that is, the smaller the possibility of belonging to the area affected by noise; therefore, there is a negative correlation between the texture tightness and the possibility of the noise area, and a positive correlation between the area swing degree and the possibility of the noise area; further, normalize the product of the negative correlation mapping value of the texture tightness and the area swing degree to determine the possibility of the noise area for each to-be-recognized segmented area in the underwater environment grayscale image. In other possible implementation manners of the embodiments of the present invention, the mean value or the sum value can also be used to replace the product for calculating the possibility of the noise area, which will not be elaborated further here.

[0071] In a specific implementation manner of the embodiments of the present invention, the process of obtaining the possibility of the noise area is represented by the formula: ; where is the possibility of the noise area for the thframe of the underwater environment grayscale image and the th to-be-recognized segmented area; is the th The texture compactness of a segmentation region to be recognized; is the degree of regional swing of the th segmentation region to be recognized in the grayscale image of the underwater environment in the th frame; is the exponential function with the natural constant as the base; is the negative correlation mapping value of the texture compactness of the th segmentation region to be recognized in the grayscale image of the underwater environment in the

[0072] Preferably, in some possible implementation manners of the embodiments of the present invention, the process of obtaining the real dirt region includes:

[0073] The greater the possibility of the noise region, the more likely the segmentation region to be recognized belongs to the region corresponding to the noise interference, that is, the smaller the possibility of belonging to the real dirt region. Therefore, by the method of threshold screening, the segmentation region to be recognized with the possibility of the noise region less than the preset noise threshold is used as the real dirt region. In a specific implementation manner of the embodiments of the present invention, the preset noise threshold is set to 0.53, which can be adjusted according to the specific implementation environment and will not be further elaborated here. After determining the real dirt region, cleaning is performed by the AUV according to the recognized real dirt region.

[0074] In summary, the present application proposes an underwater environment visual perception system for the cleaning process of an underwater robot. Based on the characteristic that the dirt region has a higher compactness relative to other noise regions, the system first determines the texture compactness according to the uniformity of the gray-scale distribution and the complexity of the gray-scale texture of each local region in each segmentation region to be recognized; further, according to the characteristic that the dirt region usually remains fixed relative to dynamic interference factors such as dynamic bubbles, attached seaweeds, and dynamic noise-like interference, the system determines the degree of regional swing representing the possibility of regional interference according to the inter-frame position change of each segmentation region to be recognized in the grayscale image of the underwater environment; further, the system comprehensively represents the possibility of the noise region according to the texture compactness and the degree of regional swing, so as to more accurately screen out the real dirt region according to the possibility of the noise region, solve the problem that the dirt region segmented by directly using the semantic segmentation method is not accurate enough, and improve the accuracy of dirt recognition in the underwater environment.

[0075] It should be noted that the above sequence of embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0076] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments.

Claims

1. An underwater environment visual perception system for underwater robot cleaning work, characterized in that: The system comprises: The data acquisition preprocessing module is used to obtain each frame of the underwater environment grayscale image during the underwater robot cleaning process in time sequence; perform semantic segmentation on the underwater environment grayscale image to obtain corresponding segmented areas to be identified; The first determination module is used to determine the texture compactness of each segmented area to be identified in each frame of the underwater environment grayscale image according to the uniformity of the grayscale distribution of each local area in each segmented area to be identified and the complexity of the grayscale texture; the texture compactness is used to indicate the distribution degree of the grayscale texture of each segmented area to be identified; A second determination module is used to determine the region swing degree of each segmented region to be identified according to the inter-frame position change of each segmented region to be identified in the underwater environment grayscale image; The underwater environment dirt recognition module is used to determine the noise area possibility of each to-be-identified segmented area in the underwater environment grayscale image according to the texture compactness and the area swing degree; screen out the real dirt area in the underwater environment grayscale image according to the noise area possibility; and perform underwater environment dirt recognition according to the real dirt area.

2. The underwater environment visual perception system for underwater robot cleaning work according to claim 1 is characterized in that: The process of obtaining the segmented area to be identified includes: The underwater environment grayscale image is input into a trained U-Net convolutional neural network, and each segmented area to be identified in the underwater environment grayscale image is output; wherein the loss function adopts a cross entropy loss function.

3. The underwater environment visual perception system for underwater robot cleaning work according to claim 1 is characterized in that: The process of obtaining the texture compactness includes: In the underwater environment grayscale image, each segmented area to be identified is divided into at least two local areas with the same shape and area; a grayscale co-occurrence matrix of each local area is constructed by a grayscale co-occurrence matrix algorithm and a corresponding energy value is calculated; the average energy value of the grayscale co-occurrence matrices of all local areas in each segmented area to be identified is used as a reference energy value of each segmented area to be identified; In each segmentation region to be identified, the energy divergence of each segmentation region to be identified is determined according to the energy value distribution disorder of the gray level co-occurrence matrix of each local region; The texture compactness of each segmented region to be identified is determined according to the energy divergence and the reference energy value; the energy divergence and the texture compactness are negatively correlated, and the reference energy value and the texture compactness are positively correlated.

4. The underwater environment visual perception system for underwater robot cleaning work according to claim 3 is characterized in that: The energy divergence acquisition process includes: The energy value variance of the gray level co-occurrence matrix of all local regions in each segmented region to be identified is taken as the energy divergence.

5. The underwater environment visual perception system for underwater robot cleaning process according to claim 3 is characterized in that: The process of determining the texture compactness of each segmented region to be identified according to the energy divergence and the reference energy value comprises: The texture compactness of each to-be-identified segmented region is determined according to the product of the negative correlation mapping value of the energy divergence and the reference energy value.

6. The underwater environment visual perception system for underwater robot cleaning work according to claim 1, characterized in that: The process of obtaining the regional swing degree includes: Counting the centroid coordinates of each to-be-identified segmented region in each frame of the underwater environment grayscale image; using other underwater environment grayscale images outside each frame of the underwater environment grayscale image as reference grayscale images for each frame of the underwater environment grayscale image; For any frame of underwater environment grayscale image: take each segmented area to be identified in the underwater environment grayscale image as the target feature area in turn; take the centroid coordinate point of the target feature area as the target coordinate point; in each reference grayscale image corresponding to the underwater environment grayscale image, take the segmented area to be identified corresponding to the centroid coordinate point with the smallest distance from the target coordinate point as the matching feature area of ​​the target feature area in each reference grayscale image; The Euclidean distance between the centroid coordinate point of the target feature area and the centroid coordinate point of the matching feature area in each corresponding reference grayscale image is used as the reference change distance corresponding to the target feature area in each reference grayscale image; the mean of the reference change distances corresponding to the target feature area in all reference grayscale images is positively correlated and mapped to determine the regional swing degree of the target feature area.

7. The underwater environment visual perception system for underwater robot cleaning work according to claim 1 is characterized in that: The process of obtaining the possibility of the noise region includes: The product of the negative correlation mapping value of the texture compactness and the area swing degree is normalized to determine the noise area possibility of each segmented area to be identified in the underwater environment grayscale image.

8. The underwater environment visual perception system for underwater robot cleaning work according to claim 7, characterized in that: The process of obtaining the real dirt area includes: The segmented area to be identified whose noise area probability is less than the preset noise threshold is regarded as the real dirt area.

9. The underwater environment visual perception system for underwater robot cleaning work according to claim 8, characterized in that: The preset noise threshold is set to 0.

53.

10. The underwater environment visual perception system for underwater robot cleaning work according to claim 3, characterized in that: The process of dividing each segmented area to be identified into at least two local areas with the same shape and area comprises: According to the 5×5 division specification, each segmented area to be identified is divided into 25 rectangular local areas of the same size.

Images