Underwater environment visual perception system for underwater robot in cleaning work process

Abstract

The invention relates to the technical field of image recognition, in particular to an underwater environment visual perception system oriented to the cleaning work process of an underwater robot. The method comprises the following steps: firstly, determining texture tightness according to a gray level distribution uniformity condition and a gray level texture complexity condition of each local area in each segmented area to be identified; further, according to the characteristic that the dirt area is generally fixed relative to interference factors such as dynamically changing bubbles, attached seaweeds and dynamically changing noise, and according to the inter-frame position change condition of each segmented area to be identified in the underwater environment grayscale image, determining the area swing degree representing the area interference possibility; furthermore, the noise region possibility is comprehensively represented according to the texture tightness and the region swing degree, so that a more accurate real dirt region is screened out according to the noise region possibility, and the accuracy of underwater environment dirt recognition is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an underwater environment visual perception system for an underwater robot during cleaning work. Background Art

[0002] The core task of the underwater environment visual perception system in the underwater robot cleaning work is to perceive and analyze the underwater environment through sensors such as cameras to achieve efficient and intelligent cleaning. Among them, underwater robots refer to automated equipment that can perform tasks in underwater environments, also known as underwater drones. In cleaning operations, autonomous underwater robots (AUVs) with autonomous navigation capabilities are usually used. They rely on preset programs or sensors to perceive the environment and complete tasks autonomously; and are usually equipped with thrusters, cameras, environmental sensors and cleaning tools such as brushes, high-pressure water guns, etc. The goal of underwater cleaning is usually to remove dirt from the surface of the hull, marine facilities, storage tanks or underwater structures. In order to determine which areas need to be cleaned, it is first necessary to identify the location corresponding to the dirt, so that the cleaning route can be planned by AUV according to the identified dirt location.

[0003] The prior art usually adopts an image recognition method, and after determining the underwater environment image, the required dirt area is segmented out by a semantic segmentation method; however, due to the turbid water quality and the high depth, the underwater light intensity is usually low, so that the clarity of the collected underwater environment image is usually poor. At this time, the underwater environment image obtained is greatly affected by other factors such as noise. If the underwater environment image is directly semantically segmented, bubbles, attached seaweed and dynamically changing noise will affect the recognition of the dirt area, making the dirt area segmented directly by the semantic segmentation method inaccurate, thereby reducing the accuracy of underwater environment dirt recognition. Summary of the invention

[0004] The present application provides an underwater environment visual perception system for underwater robot cleaning work. The system first determines the texture compactness according to the uniform grayscale distribution and grayscale texture complexity of each local area in each segmented area to be identified based on the characteristic that the dirt area has a higher compactness than other noise areas. Further, based on the characteristic that the dirt area is usually fixed relative to interference factors such as dynamically changing bubbles, attached seaweed and dynamically changing noise, the area swing degree representing the possibility of regional interference is determined according to the inter-frame position change of each segmented area to be identified in the grayscale image of the underwater environment. Further, the possibility of the noise area is comprehensively represented according to the texture compactness and the area swing degree, so as to screen out more accurate real dirt areas according to the possibility of the noise area, thereby solving the problem that the dirt area segmented directly by using the semantic segmentation method is not accurate enough, and making the accuracy of underwater environment dirt identification higher.

[0005] The present application provides an underwater environment visual perception system for an underwater robot during cleaning work, comprising: The data acquisition preprocessing module is used to obtain each frame of the underwater environment grayscale image during the underwater robot cleaning process in time sequence; perform semantic segmentation on the underwater environment grayscale image to obtain corresponding segmented areas to be identified; The first determination module is used to determine the texture compactness of each segmented area to be identified in each frame of the underwater environment grayscale image according to the uniformity of the grayscale distribution of each local area in each segmented area to be identified and the complexity of the grayscale texture; A second determination module is used to determine the region swing degree of each segmented region to be identified according to the inter-frame position change of each segmented region to be identified in the underwater environment grayscale image; The underwater environment dirt recognition module is used to determine the noise area possibility of each to-be-identified segmented area in the underwater environment grayscale image according to the texture compactness and the area swing degree; screen out the real dirt area in the underwater environment grayscale image according to the noise area possibility; and perform underwater environment dirt recognition according to the real dirt area.

[0006] Furthermore, the process of obtaining the segmented area to be identified includes: The underwater environment grayscale image is input into a trained U-Net convolutional neural network, and each segmented area to be identified in the underwater environment grayscale image is output; wherein the loss function adopts a cross entropy loss function.

[0007] Furthermore, the process of acquiring the texture compactness includes: In the underwater environment grayscale image, each segmented area to be identified is divided into at least two local areas with the same shape and area; a grayscale co-occurrence matrix of each local area is constructed by a grayscale co-occurrence matrix algorithm and a corresponding energy value is calculated; the average energy value of the grayscale co-occurrence matrices of all local areas in each segmented area to be identified is used as a reference energy value of each segmented area to be identified; In each segmentation region to be identified, the energy divergence of each segmentation region to be identified is determined according to the energy value distribution disorder of the gray level co-occurrence matrix of each local region; The texture compactness of each segmented region to be identified is determined according to the energy divergence and the reference energy value; the energy divergence and the texture compactness are negatively correlated, and the reference energy value and the texture compactness are positively correlated.

[0008] Furthermore, the process of obtaining the energy divergence includes: The energy value variance of the gray level co-occurrence matrix of all local regions in each segmented region to be identified is taken as the energy divergence.

[0009] Furthermore, the process of determining the texture compactness of each segmented region to be identified according to the energy divergence and the reference energy value includes: The texture compactness of each to-be-identified segmented region is determined according to the product of the negative correlation mapping value of the energy divergence and the reference energy value.

[0010] Furthermore, the process of obtaining the regional swing degree includes: Counting the centroid coordinates of each to-be-identified segmented region in each frame of the underwater environment grayscale image; using other underwater environment grayscale images outside each frame of the underwater environment grayscale image as reference grayscale images for each frame of the underwater environment grayscale image; For any frame of underwater environment grayscale image: take each segmented area to be identified in the underwater environment grayscale image as the target feature area in turn; take the centroid coordinate point of the target feature area as the target coordinate point; in each reference grayscale image corresponding to the underwater environment grayscale image, take the segmented area to be identified corresponding to the centroid coordinate point with the smallest distance from the target coordinate point as the matching feature area of ​​the target feature area in each reference grayscale image; The Euclidean distance between the centroid coordinate point of the target feature area and the centroid coordinate point of the matching feature area in each corresponding reference grayscale image is used as the reference change distance corresponding to the target feature area in each reference grayscale image; the mean of the reference change distances corresponding to the target feature area in all reference grayscale images is positively correlated and mapped to determine the regional swing degree of the target feature area.

[0011] Furthermore, the process of obtaining the possibility of the noise area includes: The product of the negative correlation mapping value of the texture compactness and the area swing degree is normalized to determine the noise area possibility of each segmented area to be identified in the underwater environment grayscale image.

[0012] Furthermore, the process of obtaining the real dirt area includes: The segmented area to be identified whose noise area probability is less than the preset noise threshold is regarded as the real dirt area.

[0013] Furthermore, the preset noise threshold is set to 0.53.

[0014] Furthermore, the process of dividing each to-be-identified segmented region into at least two local regions of the same shape and area includes: According to the 5×5 division specification, each segmented area to be identified is divided into 25 rectangular local areas of the same size.

[0015] This application has the following beneficial effects: First of all, the key point that needs to be dealt with in this application is to process each screened segmented area to be identified according to the difference characteristics between the dirt area and the noise area, so as to screen out the required real dirt area according to the possibility of the obtained noise area; therefore, it is necessary to analyze the difference characteristics between the dirt area and the noise area.

[0016] For the dirt attached to the surface of underwater facilities, its main visual characteristics are dense arrangement, overall coverage and compact stacking; while the main visual characteristics of the noise area corresponding to bubbles, attached seaweed and dynamically changing noise are dispersion and sparseness, such as floating algae or attached algae, bubbles, etc. Therefore, the dirt area has better compactness than the noise area, that is, the primary means of distinction is to measure the compactness of each segmented area to be identified, thereby indirectly characterizing the possibility of belonging to the noise area; therefore, the present application determines the texture compactness according to the uniformity of the grayscale distribution of each local area in each segmented area to be identified and the complexity of the grayscale texture, so that the greater the texture compactness, the more likely the corresponding segmented area to be identified belongs to the dirt area, that is, the smaller the possibility of belonging to the noise area.

[0017] It is further necessary to consider that there may be accidental circumstances that cause the visual features of some floating algae to also appear as closely arranged. Therefore, in order to reduce the impact of accidental circumstances on the identification of dirt areas, it is necessary to analyze the texture based on the compactness and other distinguishing features. For dirt attached to the surface of underwater facilities, the dirt is usually attached to the surface of the underwater facilities and will not change position over time; under the influence of water flow, algae or bubble noise will show dynamic change characteristics, specifically, algae will be affected by water flow and thus show swaying characteristics, and bubbles will produce a certain displacement under the influence of water flow; therefore, the noise area will have a significant position change in the time sequence compared to the dirt area. Therefore, the present application determines the degree of regional swing according to the inter-frame position change of each segmented area to be identified in the grayscale image of the underwater environment, so that the greater the degree of regional swing, the more likely the corresponding segmented area to be identified belongs to the noise area, that is, the less likely it is to belong to the dirt area.

[0018] Finally, the dirt noise distinction features represented by texture compactness and regional swing degree are integrated to determine the noise area possibility of each recognition feature area, so as to screen out more accurate real dirt areas in the underwater environment grayscale image according to the noise area possibility, making the underwater environment dirt recognition more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A structural block diagram of an underwater environment visual perception system for an underwater robot during cleaning work provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation method, structure, characteristics and effects of the underwater environment visual perception system for the underwater robot cleaning process proposed according to the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment, and the specific features, structures or characteristics in one or more embodiments may be combined in any suitable form. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0022] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0023] The following is a detailed description of a specific scheme of an underwater environment visual perception system for an underwater robot during cleaning work provided by the present invention in conjunction with the accompanying drawings.

[0024] The present application embodiment provides an underwater environment visual perception system for underwater robot cleaning work process, please refer to Figure 1 , which shows a structural block diagram of an underwater environment visual perception system for an underwater robot cleaning process provided by an embodiment of the present invention, the system includes a data acquisition preprocessing module 101, a first determination module 102, a second determination module 103 and an underwater environment dirt recognition module 104, wherein: The data acquisition preprocessing module 101 is used to obtain each frame of the underwater environment grayscale image during the underwater robot cleaning operation in time sequence; and perform semantic segmentation on the underwater environment grayscale image to obtain corresponding segmented areas to be identified.

[0025] In a specific implementation of the embodiment of the present invention, the underwater robot selects an AUV robot, a high-definition underwater camera is installed at the front end of the AUV robot, and the image is taken directly on the surface of the underwater facility in the underwater environment to obtain each frame of the underwater environment initial image; and in order to facilitate the subsequent grayscale analysis, the underwater environment initial image is further grayscaled to obtain each frame of the analyzed underwater environment grayscale image. Among them, the waterproof level of the high-definition underwater camera is set to IP68, and a high-definition underwater camera with a higher waterproof level can be used according to the specific implementation environment; the resolution of the underwater environment initial image is 3840×2160, and the frame rate of shooting the underwater environment initial image is set to 30fps, and the resolution size and frame rate size can be adjusted according to the specific implementation environment. It should be noted that the total number of underwater environment grayscale images analyzed in the embodiment of the present invention is set to 60, and all the analyzed underwater environment initial images are images of the same underwater facility surface taken by the high-definition underwater camera in a fixed and different manner to ensure the accuracy of the swing degree of the area analyzed later, which will not be further elaborated here.

[0026] After obtaining the grayscale image of the underwater environment, the semantic segmentation method is first used to determine the various segmented areas to be identified that need to be analyzed in the embodiment of the present invention; the segmented areas to be identified may be the required dirt areas, or they may be noise areas corresponding to bubbles, attached seaweed, and dynamically changing noise. Therefore, it is necessary to further analyze the segmented areas to be identified and screen out the required real dirt areas.

[0027] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring the segmented region to be identified includes: The underwater environment grayscale image is input into the trained U-Net convolutional neural network, and each segmented area to be identified in the underwater environment grayscale image is output; wherein the loss function adopts the cross entropy loss function. It should be noted that the U-Net convolutional neural network is a semantic segmentation technology well known to those skilled in the art, and implementers may adopt other methods according to the specific implementation environment, which will not be further limited or elaborated here.

[0028] The first determination module 102 is used to determine the texture compactness of each segmented area to be identified in each frame of the underwater environment grayscale image according to the uniformity of the grayscale distribution of each local area in each segmented area to be identified and the complexity of the grayscale texture.

[0029] First of all, the key point that needs to be dealt with in this application is to process each screened segmented area to be identified according to the difference characteristics between the dirt area and the noise area, so as to screen out the required real dirt area according to the possibility of the obtained noise area; therefore, it is necessary to analyze the difference characteristics between the dirt area and the noise area.

[0030] For the dirt attached to the surface of underwater facilities, its main visual characteristics are dense arrangement, overall coverage and compact stacking; while the main visual characteristics of the noise area corresponding to bubbles, attached seaweed and dynamically changing noise are dispersion and sparseness, such as floating algae or attached algae, bubbles, etc. Therefore, the dirt area has better compactness than the noise area, that is, the primary means of distinction is to measure the compactness of each segmented area to be identified, thereby indirectly characterizing the possibility of belonging to the noise area; therefore, the present application determines the texture compactness according to the uniformity of the grayscale distribution of each local area in each segmented area to be identified and the complexity of the grayscale texture, so that the greater the texture compactness, the more likely the corresponding segmented area to be identified belongs to the dirt area, that is, the smaller the possibility of belonging to the noise area.

[0031] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring texture compactness includes: In the grayscale image of the underwater environment, each segmented area to be identified is divided into at least two local areas with the same shape and area; the grayscale co-occurrence matrix of each local area is constructed by the grayscale co-occurrence matrix algorithm and the corresponding energy value is calculated; the mean energy value of the grayscale co-occurrence matrices of all local areas in each segmented area to be identified is used as the reference energy value of each segmented area to be identified. For the grayscale co-occurrence matrix, the higher the energy value, the more uniform or consistent the texture of the corresponding image area; therefore, the larger the mean energy value of the grayscale co-occurrence matrix of all local areas of the segmented area to be identified, that is, the larger the reference energy value, the more uniform the local grayscale texture distribution of the segmented area to be identified is in the dimension of local analysis, that is, the more consistent with the characteristics of the dense texture distribution of the dirt area. It should be noted that the grayscale co-occurrence matrix is ​​a technical means well known to those skilled in the art, and will not be further defined or elaborated here.

[0032] In a specific implementation of the present embodiment, the process of dividing each segmented area to be identified into at least two local areas of the same shape and area includes: dividing each segmented area to be identified into 25 local areas of the same size and rectangular shape according to a 5×5 division specification. It should be noted that the implementer may use other methods or specifications to divide the segmented area to be identified according to the specific implementation environment, which will not be further described here.

[0033] In each segmented area to be identified, the energy divergence of each segmented area to be identified is determined according to the chaotic distribution of the energy values ​​of the gray level co-occurrence matrix of each local area. Since the energy value represents the texture uniformity of each local area, if the overall distribution of the segmented area to be identified is relatively tight, the texture uniformity of each local area should be relatively concentrated and relatively uniform as a whole. Therefore, the greater the degree of energy value divergence obtained through the chaotic distribution of energy values, the more discrete the distribution of texture uniformity of each local area, and the less the segmented area to be identified meets the characteristics of the dense texture distribution of the dirt area.

[0034] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring energy divergence includes: The energy value variance of the gray level co-occurrence matrix of all local regions in each segmented region to be identified is taken as the energy divergence. For any set of data, the larger the variance, the more discrete the corresponding data. Therefore, the energy divergence is determined according to the energy value variance of the gray level co-occurrence matrix of all local regions, so that when the energy divergence is larger, the energy value distribution of the gray level co-occurrence matrix of all local regions is more discrete, that is, the corresponding segmented region to be identified is less consistent with the feature of the relatively tight texture distribution of the dirt region.

[0035] The energy divergence and the reference energy value are further combined, and the texture compactness of each segmented area to be identified is determined according to the energy divergence and the reference energy value; since the smaller the energy divergence and the larger the reference energy value, the more the corresponding segmented area to be identified conforms to the characteristics of the relatively compact texture distribution of the dirt area, that is, the larger the texture compactness, the more likely the corresponding segmented area to be identified is a dirt area, so the energy divergence and the texture compactness are negatively correlated, and the reference energy value and the texture compactness are positively correlated.

[0036] Preferably, in some possible implementations of the embodiments of the present invention, the process of determining the texture compactness of each to-be-identified segmented region according to the energy divergence and the reference energy value includes: According to the product between the negative correlation mapping value of the energy divergence and the reference energy value, the texture compactness of each segmented region to be identified is determined. It should be noted that in addition to the product, the implementer can also calculate the texture compactness by other methods according to the correlation, such as the normalized value of the mean and the sum value, etc., which will not be further elaborated here.

[0037] In a specific implementation of the embodiment of the present invention, the process of obtaining texture compactness is expressed by the formula: ;in, For the The first grayscale image of the underwater environment The texture compactness of the segmented area to be identified; For the The first grayscale image of the underwater environment The energy value variance of the gray level co-occurrence matrix of all local regions in the segmented region to be identified, that is, the energy divergence; For the The first grayscale image of the underwater environment The number of local regions in the segmented region to be identified; For the The first grayscale image of the underwater environment The first segmentation area to be identified The energy value of the gray-level co-occurrence matrix of a local area; For the The first grayscale image of the underwater environment The mean energy value of the gray level co-occurrence matrix of all local areas in the segmentation area to be identified, that is, the reference energy value; is an exponential function with a natural constant as its base.

[0038] The second determination module 103 is used to determine the region swing degree of each segmented region to be identified according to the inter-frame position change of each segmented region to be identified in the underwater environment grayscale image.

[0039] It is further necessary to consider that there may be accidental circumstances that cause the visual features of some floating algae to also appear as closely arranged. Therefore, in order to reduce the impact of accidental circumstances on the identification of dirt areas, it is necessary to analyze the texture based on the compactness and other distinguishing features. For dirt attached to the surface of underwater facilities, the dirt is usually attached to the surface of the underwater facilities and will not change position over time; under the influence of water flow, algae or bubble noise will show dynamic change characteristics, specifically, algae will be affected by water flow and thus show swaying characteristics, and bubbles will produce a certain displacement under the influence of water flow; therefore, the noise area will have a significant position change in the time sequence compared to the dirt area. Therefore, the present application determines the degree of regional swing according to the inter-frame position change of each segmented area to be identified in the grayscale image of the underwater environment, so that the greater the degree of regional swing, the more likely the corresponding segmented area to be identified belongs to the noise area, that is, the less likely it is to belong to the dirt area.

[0040] Preferably, in some possible implementations of the embodiments of the present invention, the process of acquiring the regional swing degree includes: The centroid coordinate points of each segmented area to be identified in each frame of the underwater environment grayscale image are counted; other underwater environment grayscale images outside each frame of the underwater environment grayscale image are used as reference grayscale images for each frame of the underwater environment grayscale image. Among them, the centroid coordinate points are used to measure the position of the segmented area to be identified, and other coordinate points such as the centroid coordinate points can also be used for analysis, which will not be further elaborated here.

[0041] For any frame of underwater environment grayscale image: take each to-be-identified segmented area in the underwater environment grayscale image as the target feature area in turn; take the centroid coordinate point of the target feature area as the target coordinate point; in each reference grayscale image corresponding to the underwater environment grayscale image, take the to-be-identified segmented area corresponding to the centroid coordinate point with the smallest distance to the target coordinate point as the matching feature area of ​​the target feature area in each reference grayscale image.

[0042] For the target feature region, if it belongs to the region corresponding to algae or bubble noise, due to the corresponding dynamic change characteristics, the target feature region will produce a certain position change in different frames of underwater environment grayscale images; while the real dirt will not be affected by the water flow, so that the target feature region corresponding to the real dirt in different frames of underwater environment grayscale images usually does not change position or changes slightly; therefore, for the target feature region, the smaller the overall distance between its corresponding target coordinate point and the centroid coordinate point of each matching feature region, the smaller the position change of the target feature region in different frames of underwater environment grayscale images, and the more it conforms to the feature that the position of the dirt region remains unchanged. Therefore, the Euclidean distance between the centroid coordinate point of the target feature region and the centroid coordinate point of the matching feature region in each reference grayscale image is further used as the reference change distance corresponding to the target feature region in each reference grayscale image; the mean of the reference change distance corresponding to the target feature region in all reference grayscale images is positively correlated and mapped to determine the regional swing degree of the target feature region. The smaller the mean of the corresponding reference change distances in all reference grayscale images, that is, the smaller the degree of regional swing, the more likely the corresponding target feature area is to belong to the area corresponding to dirt, that is, the smaller the possibility of belonging to the noise-affected area.

[0043] In a specific implementation of the embodiment of the present invention, the process of obtaining the regional swing degree is expressed by the formula: ;in, For the The first grayscale image of the underwater environment The degree of regional swing of the segmented area to be identified; is the total number of grayscale images of underwater environment; For the The first grayscale image of the underwater environment The centroid coordinate point of the segmented area to be identified and the corresponding The Euclidean distance between the centroid coordinate points of the matching feature regions in the reference grayscale image, also known as the reference change distance; The preset positive correlation mapping parameter is set to 0.1 in the embodiment of the present invention to prevent the calculated area swing degree from being equal to 0, thereby reducing the impact on the subsequent noise area possibility calculation. The size of the preset positive correlation mapping parameter can be adjusted according to the specific implementation environment; It is a linear normalization function. The implementer can adjust the normalization method according to the specific implementation environment.

[0044] The underwater environment dirt recognition module 104 is used to determine the noise area possibility of each to-be-identified segmented area in the underwater environment grayscale image according to the texture compactness and the area swing degree; screen out the real dirt area in the underwater environment grayscale image according to the noise area possibility; and perform underwater environment dirt recognition according to the real dirt area.

[0045] Finally, the dirt noise distinction features represented by texture compactness and regional swing degree are integrated to determine the noise area possibility of each recognition feature area, so as to screen out more accurate real dirt areas in the underwater environment grayscale image according to the noise area possibility, making the underwater environment dirt recognition more accurate.

[0046] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the possibility of the noise area includes: Since the greater the texture compactness and the smaller the regional swing, the more likely the corresponding segmented area to be identified is a dirt area, that is, the smaller the possibility of the corresponding area being affected by noise; therefore, the texture compactness and the possibility of the noise area are negatively correlated, and the regional swing degree and the possibility of the noise area are positively correlated; further, the product between the negative correlation mapping value of the texture compactness and the regional swing degree is normalized to determine the noise area possibility of each segmented area to be identified in the grayscale image of the underwater environment. In other possible implementation methods of the embodiments of the present invention, the mean or sum value can also be used instead of the product to calculate the noise area possibility, which will not be further elaborated here.

[0047] In a specific implementation of the embodiment of the present invention, the process of obtaining the possibility of the noise area is expressed by the formula: ;in, For the The first grayscale image of the underwater environment The possibility of noise areas in the segmented areas to be identified; For the The first grayscale image of the underwater environment The texture compactness of the segmented area to be identified; For the The first grayscale image of the underwater environment The degree of regional swing of the segmented area to be identified; is an exponential function with a natural constant as base; For the The first grayscale image of the underwater environment A negative correlation mapping value of the texture compactness of the segmented area to be identified; It is a linear normalization function. The implementer can adjust the normalization method according to the specific implementation environment. The normalization method limits the possible value range of the noise area to 0 to 1, which is convenient for the subsequent screening of the real dirt area.

[0048] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the real dirt area includes: The greater the possibility of the noise area, the more likely the segmented area to be identified belongs to the area corresponding to the noise interference, that is, the smaller the possibility of belonging to the real dirt area. Therefore, through the threshold screening method, the segmented area to be identified whose noise area possibility is less than the preset noise threshold is taken as the real dirt area. In a specific implementation of an embodiment of the present invention, the preset noise threshold is set to 0.53, which can be adjusted according to the specific implementation environment, and will not be further described here. After determining the real dirt area, the AUV is used to clean according to the identified real dirt area.

[0049] In summary, the present application proposes an underwater environment visual perception system for underwater robot cleaning work. The system first determines the texture compactness based on the uniform grayscale distribution of each local area in each segmented area to be identified and the complexity of the grayscale texture, based on the characteristic that the dirt area has a higher compactness than other noise areas; further, based on the characteristic that the dirt area is usually fixed relative to interference factors such as dynamically changing bubbles, attached seaweed and dynamically changing noise, the degree of regional swing that represents the possibility of regional interference is determined according to the inter-frame position change of each segmented area to be identified in the grayscale image of the underwater environment; further, the possibility of the noise area is comprehensively characterized based on the texture compactness and the degree of regional swing, so as to screen out more accurate real dirt areas based on the possibility of the noise area, thereby solving the problem that the dirt area segmented directly by the semantic segmentation method is not accurate enough, and making the accuracy of underwater environment dirt recognition higher.

[0050] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0051] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. An underwater environment visual perception system for underwater robot cleaning work, characterized in that: The system comprises: The data acquisition preprocessing module is used to obtain each frame of the underwater environment grayscale image during the underwater robot cleaning process in time sequence; perform semantic segmentation on the underwater environment grayscale image to obtain corresponding segmented areas to be identified; The first determination module is used to determine the texture compactness of each segmented area to be identified in each frame of the underwater environment grayscale image according to the uniformity of the grayscale distribution of each local area in each segmented area to be identified and the complexity of the grayscale texture; A second determination module is used to determine the region swing degree of each segmented region to be identified according to the inter-frame position change of each segmented region to be identified in the underwater environment grayscale image; The underwater environment dirt recognition module is used to determine the noise area possibility of each to-be-identified segmented area in the underwater environment grayscale image according to the texture compactness and the area swing degree; screen out the real dirt area in the underwater environment grayscale image according to the noise area possibility; and perform underwater environment dirt recognition according to the real dirt area.

2. The underwater environment visual perception system for underwater robot cleaning work according to claim 1 is characterized in that: The process of obtaining the segmented area to be identified includes: The underwater environment grayscale image is input into a trained U-Net convolutional neural network, and each segmented area to be identified in the underwater environment grayscale image is output; wherein the loss function adopts a cross entropy loss function.

3. The underwater environment visual perception system for underwater robot cleaning work according to claim 1 is characterized in that: The process of obtaining the texture compactness includes: In the underwater environment grayscale image, each segmented area to be identified is divided into at least two local areas with the same shape and area; a grayscale co-occurrence matrix of each local area is constructed by a grayscale co-occurrence matrix algorithm and a corresponding energy value is calculated; the average energy value of the grayscale co-occurrence matrices of all local areas in each segmented area to be identified is used as a reference energy value of each segmented area to be identified; In each segmentation region to be identified, the energy divergence of each segmentation region to be identified is determined according to the energy value distribution disorder of the gray level co-occurrence matrix of each local region; The texture compactness of each segmented region to be identified is determined according to the energy divergence and the reference energy value; the energy divergence and the texture compactness are negatively correlated, and the reference energy value and the texture compactness are positively correlated.

4. The underwater environment visual perception system for underwater robot cleaning work according to claim 3 is characterized in that: The energy divergence acquisition process includes: The energy value variance of the gray level co-occurrence matrix of all local regions in each segmented region to be identified is taken as the energy divergence.

5. The underwater environment visual perception system for underwater robot cleaning process according to claim 3 is characterized in that: The process of determining the texture compactness of each segmented region to be identified according to the energy divergence and the reference energy value comprises: The texture compactness of each to-be-identified segmented region is determined according to the product of the negative correlation mapping value of the energy divergence and the reference energy value.

6. The underwater environment visual perception system for underwater robot cleaning work according to claim 1, characterized in that: The process of obtaining the regional swing degree includes: Counting the centroid coordinates of each to-be-identified segmented region in each frame of the underwater environment grayscale image; using other underwater environment grayscale images outside each frame of the underwater environment grayscale image as reference grayscale images for each frame of the underwater environment grayscale image; For any frame of underwater environment grayscale image: take each segmented area to be identified in the underwater environment grayscale image as the target feature area in turn; take the centroid coordinate point of the target feature area as the target coordinate point; in each reference grayscale image corresponding to the underwater environment grayscale image, take the segmented area to be identified corresponding to the centroid coordinate point with the smallest distance from the target coordinate point as the matching feature area of ​​the target feature area in each reference grayscale image; The Euclidean distance between the centroid coordinate point of the target feature area and the centroid coordinate point of the matching feature area in each corresponding reference grayscale image is used as the reference change distance corresponding to the target feature area in each reference grayscale image; the mean of the reference change distances corresponding to the target feature area in all reference grayscale images is positively correlated and mapped to determine the regional swing degree of the target feature area.

7. The underwater environment visual perception system for underwater robot cleaning work according to claim 1 is characterized in that: The process of obtaining the possibility of the noise region includes: The product of the negative correlation mapping value of the texture compactness and the area swing degree is normalized to determine the noise area possibility of each segmented area to be identified in the underwater environment grayscale image.

8. The underwater environment visual perception system for underwater robot cleaning work according to claim 7, characterized in that: The process of obtaining the real dirt area includes: The segmented area to be identified whose noise area probability is less than the preset noise threshold is regarded as the real dirt area.

9. The underwater environment visual perception system for underwater robot cleaning work according to claim 8, characterized in that: The preset noise threshold is set to 0.

53.

10. The underwater environment visual perception system for underwater robot cleaning work according to claim 3, characterized in that: The process of dividing each segmented area to be identified into at least two local areas with the same shape and area comprises: According to the 5×5 division specification, each segmented area to be identified is divided into 25 rectangular local areas of the same size.

Images