Identification method and identification system for underwater suspicious facilities influencing water quality

Abstract

The embodiment of the invention provides a recognition method and a recognition system for underwater suspicious facilities influencing water quality. The method for identifying the underwater suspicious facilities affecting the water quality comprises the steps that a sonar point cloud obtained by detecting the underwater environment through a sonar system installed on an underwater robot is obtained, and the sonar point cloud is processed to determine the appearance characteristics of an underwater object; determining whether the underwater objects include suspicious objects based on the appearance features; controlling a visible light imaging system installed on the underwater robot to shoot the underwater environment to obtain a visible light image under the condition of determining that the underwater object comprises the suspicious object, and processing the visible light image to determine the appearance characteristics of the suspicious object; and identifying whether the suspicious object is an underwater suspicious facility influencing the water quality based on the appearance characteristics. As the appearance features have more information content than the appearance features, the types of the suspicious objects can be reliably determined by adopting the appearance features, and then whether the suspicious objects are underwater facilities affecting the water quality or not can be determined.

Description

Technical Field

[0001] The present invention relates to the technical field of water environment inspection and monitoring, and in particular to a method and system for identifying suspicious underwater facilities that affect water quality. Background Art

[0002] Water quality inspection and monitoring is one of the important tasks of environmental authorities at all levels. In order to detect water pollution and water quality fraud at the source and to achieve safety and efficiency in water quality monitoring, environmental authorities have begun to use underwater robots equipped with sonar devices to detect the water environment (more specifically, the underwater environment), construct sonar point clouds through sonar scanning, and then process the sonar point clouds to determine whether there are suspicious underwater facilities in the water that affect the water quality.

[0003] However, due to the large number of organisms in rivers and offshore areas and the large number of various sedimentary objects, the bottom morphology of the water body is diverse, and the sonar point cloud formed by the aforementioned underwater robot is complex. Due to the limitations of the subsequent data processing algorithm, it is not possible to reliably determine whether the objects in the water body are sewage-related facilities based on the sonar point cloud. Summary of the invention

[0004] In order to solve the problem that the existing sonar point cloud-based method cannot reliably determine whether an object in a water body is a sewage-related facility, the embodiments of the present disclosure provide a method and system for identifying suspicious underwater facilities that affect water quality.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for identifying suspicious underwater facilities that affect water quality, comprising:

[0006] Acquire a sonar point cloud obtained by a sonar system installed on the underwater robot to detect the underwater environment, and process the sonar point cloud to determine the shape characteristics of the underwater object;

[0007] Determining whether the underwater object includes a suspicious object based on the appearance features;

[0008] In the case where it is determined that the underwater object includes a suspicious object, controlling a visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image, and processing the visible light image to determine the appearance characteristics of the suspicious object;

[0009] Based on the appearance features, it is identified whether the suspicious object is a suspicious underwater facility that affects water quality.

[0010] Optionally, the controlling a visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image also includes:

[0011] Controlling the visible light imaging system to photograph the underwater environment to obtain a preliminary visible light image;

[0012] Determining whether the underwater light intensity reaches the clear imaging light intensity based on the preliminary visible light image;

[0013] When the underwater light intensity reaches the clear imaging light intensity, taking the preliminary visible light image as the visible light image; and,

[0014] When the underwater light intensity does not reach the clear imaging light intensity, the automatic light filling device is controlled to fill light into the underwater environment, and at the same time, the visible light imaging system is controlled to photograph the underwater environment to obtain the visible light image.

[0015] Optionally, in the case where it is determined that the underwater object includes a suspicious object, the method further comprises: determining an estimated size of the suspicious object based on the sonar point cloud, and determining a relative position between the suspicious object and the underwater robot based on the sonar point cloud;

[0016] Before controlling the visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image, the method further includes: determining a posture adjustment parameter of the underwater robot and / or determining an adjustment internal parameter of a shooting lens in the visible light imaging system based on the estimated size and the relative position;

[0017] The method of controlling a visible light imaging system installed on an underwater robot to photograph an underwater environment to obtain a visible light image comprises: when the underwater robot adjusts its posture position according to the posture adjustment parameters and the shooting lens is adjusted according to the adjustment internal parameters, controlling the visible light imaging system to photograph the underwater environment to obtain the visible light image.

[0018] Optionally, the shooting lens is a lens that is adjustable relative to the underwater robot body, and the method further includes:

[0019] Based on the estimated size and the relative position, determine the posture adjustment parameters of the underwater robot and / or determine the adjustment internal parameters of the shooting lens, and determine the adjustment external parameters of the shooting lens;

[0020] After the underwater robot adjusts its position according to the position adjustment parameter, and the shooting lens is adjusted according to the adjustment internal parameter, controlling the visible light imaging system to shoot the underwater environment to obtain the visible light image, including:

[0021] When the underwater robot adjusts its posture position according to the posture adjustment parameters, and the shooting lens is adjusted according to the adjustment internal parameters, and the shooting lens is adjusted according to the adjustment external parameters, the visible light imaging system is controlled to shoot the underwater environment to obtain the visible light image.

[0022] Optionally, the processing of the visible light image to determine the appearance features of the suspicious object includes:

[0023] Determining a posture adjustment feature of the shooting lens based on the posture adjustment parameter and the adjustment extrinsic parameter;

[0024] Determining the imaging viewing angle scaling feature of the shooting lens based on the adjustment internal parameter;

[0025] Determine a local image of the suspicious object in the visible light image based on the posture adjustment feature and the imaging view scaling feature, the position of the suspicious object in the sonar point cloud, the relative position, and the coordinate system transformation relationship between the sonar system and the visible light imaging system;

[0026] The partial image is processed to determine the appearance features of the suspicious object.

[0027] Optionally, the processing of the visible light image to determine the appearance features of the suspicious object includes:

[0028] Performing preliminary recognition on the visible light image to determine whether the visible light image is an underwater image with non-muddy water features;

[0029] In the case where it is determined that the visible light image is an underwater image with non-muddy water characteristics, the visible light image is processed to determine the appearance characteristics of the suspicious object.

[0030] Optionally, the identifying whether the suspicious object is a suspicious underwater facility that affects water quality based on the appearance feature includes:

[0031] Based on the shape features and the appearance features, it is identified whether the suspicious object is a suspicious underwater facility that affects water quality.

[0032] Optionally, the identifying whether the suspicious object is a suspicious underwater facility that affects water quality based on the shape feature and the appearance feature includes:

[0033] In the case where the appearance feature is a feature of a suspicious underwater facility that affects water quality, obtaining a first certainty probability;

[0034] In the case where the appearance feature is a feature of a suspicious underwater facility that affects water quality, obtaining a second certainty probability;

[0035] Determining, based on the first certainty probability and the second certainty probability, a probability of eliminating the possibility that the suspicious object is not a suspicious underwater facility that affects water quality;

[0036] When the exclusion probability is less than the set probability, the suspicious object is determined to be a suspicious underwater facility that affects water quality.

[0037] Optionally, after processing the visible light image to determine the appearance features of the suspicious object, the method further includes:

[0038] The visible light image is stored, and / or the visible light image is output through a display device.

[0039] In a second aspect, an embodiment of the present disclosure provides a system for identifying suspicious underwater facilities that affect water quality, comprising a computing device and an underwater robot; the underwater robot comprises a robot body, and a sonar system and a visible light imaging system arranged on the robot body; the computing device is used to determine whether an underwater object has any suspicious underwater facilities that affect water quality based on a sonar point cloud obtained by detecting the bottom of the water by the sonar system, in accordance with the method for identifying suspicious underwater facilities that affect water quality as described above.

[0040] The technical solution of the disclosed embodiment determines whether an underwater object is a suspicious object based on the appearance features determined by the sonar point cloud. After determining that the underwater object is a suspicious object, a visible light image is obtained by visible light imaging, and after the appearance features of the suspicious object are determined based on the visible light image, it is determined whether it is an underwater suspicious facility that affects water quality based on the appearance features. Because appearance features have more information content than appearance features, the type of suspicious object can be reliably determined by using appearance features. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor, including

[0043] Figure 1 is a flow chart of a method for identifying suspicious underwater facilities that affect water quality provided by an embodiment of the present disclosure;

[0044] Figure 2 is a flow chart of controlling an underwater robot to shoot and form visible light images in some embodiments of the present disclosure;

[0045] Figure 3 is a flow chart of a method for determining adjustment parameters of an underwater robot provided in some embodiments of the present disclosure;

[0046] Figure 4 is a schematic diagram of the structure of a device for identifying suspicious underwater facilities that affect water quality provided by an embodiment of the present disclosure;

[0047] Figure 5 It is a structural schematic diagram of a system for identifying suspicious underwater facilities that affect water quality provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0049] The term "including" and its variations used in this document are open-ended, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. In this document, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are the same.

[0050] In order to solve the problem that the existing sonar point cloud and supporting data processing methods cannot reliably determine whether there are sewage-related facilities in the water body, the embodiments of the present disclosure provide a new method for identifying suspicious underwater facilities that affect water quality. The new method for identifying suspicious underwater facilities that affect water quality provided by the embodiments of the present disclosure is executed by a computing device. In specific implementations, the computing device can be a local processing device deployed in an underwater robot, or a computing device that can exchange information with an underwater robot through data communication, or a combination of the aforementioned local processing device and computing device.

[0051] Figure 1 : is a flow chart of a method for identifying suspicious underwater facilities that affect water quality provided by an embodiment of the present disclosure. Figure 1 As shown, the method for identifying suspicious underwater facilities that affect water quality provided by the embodiment of the present disclosure includes S110-S140.

[0052] S110: Acquire a sonar point cloud obtained by a sonar system installed on the underwater robot to detect the underwater environment, and process the sonar point cloud to determine the shape characteristics of the underwater object.

[0053] When the underwater robot is lowered into the water body to detect the underwater environment, the sonar system on the underwater robot is turned on and runs, detects underwater obstacles by sound wave detection, generates a sonar point cloud based on the received reflected sound waves, and sends the sonar point cloud to the computing device. The sonar system in the embodiment of the present disclosure can be a multi-beam sonar system, which can quickly generate a sonar point cloud of a local area by multi-beam line scanning.

[0054] After receiving the sonar point cloud data, the computing device performs feature recognition on the sonar point cloud data to determine the underwater object being identified and the shape features of the underwater object. In a specific implementation, the computing device can use a pre-trained convolutional neural network point cloud recognition model to process the sonar point cloud to determine the shape features of the underwater object. The aforementioned point cloud recognition model can use the data obtained by multiple sonar systems as a training set, and use the output of the Bayesian filter model as the true value to train the initial model. Please note here that what is determined by the sonar point cloud is the shape features of the underwater object, that is, the features of the surface shape, shape and size of the underwater object.

[0055] S120: Determine whether the underwater object includes a suspicious object based on the appearance characteristics; if so, execute S130.

[0056] After obtaining the shape features of the underwater object, the computing device can compare it with various predetermined morphological features to determine the type of the underwater object and the probability of identification, and based on the type of the underwater object and the object types included in the predetermined set of suspicious objects, determine whether the underwater object includes suspicious objects.

[0057] For example, underwater objects that may appear in an underwater environment (mostly the bottom of the water) include stones, glass bottles, metal waste, drainage facilities (including drainage culverts and drainage pipes), etc. In a specific application, the shape features obtained through the sonar point cloud include features that characterize the shape of the glass bottle, and the underwater object is determined to be a glass bottle accordingly. Because the glass bottle is not in the collection of suspicious objects, the glass bottle is not a suspicious object. In another specific application, the shape features obtained through the sonar point cloud include features that characterize the drainage pipe, and the underwater object is determined to be a drainage pipe accordingly. Because the drainage pipe is in the collection of suspicious objects, the drainage pipe is a suspicious object.

[0058] S130: Control the visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image, and process the visible light image to determine the appearance characteristics of the suspicious object.

[0059] A visible light imaging system is an imaging system that forms an image by collecting visible light using an imaging device. In simple terms, a visible light imaging system is an imaging system widely used in the prior art with a camera or a camera as the core.

[0060] After determining that the underwater object includes a suspicious object, the computing device controls the visible light imaging system to shoot the underwater environment to obtain a visible light image of the underwater environment. The aforementioned visible light image can be an image or a video.

[0061] It should be noted here that the visible light image formed by the visible light imaging system photographing the underwater environment is an image including pixel information of the suspicious object, so as to ensure that the visible light image can be processed subsequently to determine the appearance characteristics of the suspicious object.

[0062] In the disclosed embodiment, the computing device can use a pre-trained convolutional neural network image processing model to process visible light images to determine the appearance characteristics of underwater objects. Because the appearance characteristics are obtained based on visible light images, the appearance characteristics are the characteristics of the shape or material of the suspicious object at the visible light level.

[0063] In order to avoid confusion between appearance features and the aforementioned shape features, an explanation of appearance features is given here. The appearance features in the implementation of the present disclosure are features of attribute characteristics including the shape of an object, surface color characteristics, material characteristics, etc. According to the aforementioned explanation, shape features include shape features. Accordingly, because appearance features have more attributes, the use of appearance features can more accurately determine the type of a suspicious object than the use of shape features.

[0064] S140: Identify whether the suspicious object is an underwater suspicious facility that affects water quality based on appearance features.

[0065] After obtaining the appearance features, it is then possible to directly identify whether the suspicious object is an underwater suspicious facility that affects water quality based on the appearance features. For example, after the aforementioned method is used to determine that the appearance features are features that characterize drainage pipes and the suspicious object is identified as a drainage pipe, and the drainage pipe is an identified underwater suspicious facility that affects water quality, it can also be determined that the suspicious object is an underwater suspicious facility that affects water quality.

[0066] It should be noted here that suspicious underwater facilities that affect water quality may be sewage facilities, facilities for local covert monitoring of water quality, facilities for optimizing water quality in local underwater areas, or supporting facilities used to assist the aforementioned facilities in realizing their functions and concealing the aforementioned facilities.

[0067] The technical solution of the disclosed embodiment takes into account the problem that the sonar point cloud obtained due to the complex underwater environment is complex in form and cannot reliably determine whether the underwater object is a suspicious underwater facility that affects the water quality. It does not directly determine whether the underwater object is a suspicious underwater facility that affects the water quality based on the appearance features determined by the sonar point cloud, but determines whether the underwater problem is a suspicious object such as a drainage pipe based on the appearance features. After determining that the underwater object is a suspicious object, a visible light image is subsequently obtained by visible light imaging. After determining the appearance features of the suspicious object based on the visible light image, it is then determined whether it is a suspicious underwater facility that affects the water quality based on the appearance features.

[0068] As analyzed above, because appearance features have more information content than shape features, the type of suspicious object can be reliably determined by using appearance features. In addition, because the shape features determined by the sonar point cloud have been used to determine that the underwater object is a suspicious object, that is, this solution uses appearance features as the front judgment basis and uses appearance features as the back judgment basis to determine whether the underwater object is an underwater suspicious facility that affects water quality through a two-step judgment method.

[0069] In some embodiments, in order to more accurately determine whether the suspicious object is a suspicious underwater facility that affects water quality, the aforementioned S140 may specifically be the following S141.

[0070] S141: Identify whether the suspicious object is an underwater suspicious facility that affects water quality based on shape features and appearance features.

[0071] The computing device determines whether the underwater object is a suspicious underwater facility that affects the water quality based on the shape features and appearance features. It is based on the mutual verification of the shape features and appearance features and the joint confirmation of the type of the suspicious object, and then determines that the suspicious object is a sewage-related facility under the tree, such as a drainage pipe.

[0072] In a specific implementation, considering that the aforementioned shape features and appearance features are data obtained by processing two unrelated raw data (sonar point cloud and visible light image), the shape features and appearance features are determined independently of each other. Therefore, the computing device can use the following S1411-S1414 to determine whether the suspicious object is an underwater suspicious facility that affects the water quality.

[0073] S1411: When the appearance feature is a feature of a suspicious underwater facility that affects water quality, obtain a first certainty probability.

[0074] When the sonar point cloud is processed by the aforementioned convolutional neural network or other processing model to obtain the shape features, the probability of the shape features being various types of objects is determined, and the object type corresponding to the maximum probability is used as the determined object type. Accordingly, if the underwater object is determined to be a suspicious underwater facility that affects the water quality through the shape features, and the corresponding shape features are the underwater suspicious facilities that characterize the water quality, the aforementioned maximum probability can be directly used as the first certainty probability.

[0075] S1412: When the appearance feature is a feature of a suspicious underwater facility that affects water quality, obtain a second certainty probability.

[0076] According to the aforementioned method for S1411 and the corresponding analysis, the second certainty probability can also be obtained.

[0077] As analyzed above, because the aforementioned shape features and appearance features are obtained by two different methods and are independent of each other, they can be processed based on independent event processing strategies to determine whether the suspicious object is an underwater suspicious facility that affects water quality.

[0078] S1413: Based on the first certainty probability and the second certainty probability, determine the exclusion probability that the suspicious object is not an underwater suspicious facility that affects water quality.

[0079] S1414: When the exclusion probability is less than the set probability, the suspicious object is determined to be an underwater suspicious facility that affects the water quality.

[0080] According to the principle of independent events in probability theory, the product of the inverse event probabilities of two independent events represents the probability that any event will not occur in the two independent events. Accordingly, based on the aforementioned first certain probability and second certain probability, the first inverse probability and second inverse probability of using two methods to determine that the suspicious object is not an underwater suspicious facility that affects water quality can be determined respectively. Multiplying the first inverse probability and the second inverse probability can obtain the exclusion probability of comprehensively judging that the suspicious object is not an underwater suspicious facility that affects water quality.

[0081] In this public embodiment, the set probability is a pre-set smaller value. If the exclusion probability is less than the set probability, the probability of determining that the object is a suspicious underwater facility that affects water quality is very small. Because a suspicious object can only be a suspicious underwater facility that affects water quality or a suspicious underwater facility that does not affect water quality, the reverse is that the probability of determining that the suspicious object is a suspicious underwater facility that affects water quality is very high, and it is directly determined to be a suspicious underwater facility that affects water quality.

[0082] As analyzed above, the underwater robot operates in an underwater environment. In specific implementation, the underwater robot may perform detection operations in water two meters or deeper. In the aforementioned underwater environment, which may be a low-light environment, the visible light imaging system of the underwater robot may not be able to form images correctly. Even if it can form a visible light image correctly, the contrast is not obvious and the image cannot fully reflect various grayscale features.

[0083] Figure 2 1 is a flow chart of controlling an underwater robot to shoot and form visible light images in some embodiments of the present disclosure. Figure 2 As shown, in some embodiments, the computing device forms a visible light image according to the following S131-S134.

[0084] S131: Control the visible light imaging system to capture the underwater environment to obtain a preliminary visible light image.

[0085] In the disclosed embodiment, when the computing device determines through S120 that the underwater object includes a suspicious object, it triggers the visible light imaging system to capture and form a preliminary visible light image. The preliminary visible light image is an image formed without fill light, that is, when the automatic fill light device described later is not turned on to illuminate the direction of the suspicious object. There is no fill light here for the following reasons: (1) The light conditions in the underwater environment may be low and fill light is not required; (2) If fill light is forced, strong light reflection from the underwater environment may occur, forming a highlight area, and it is impossible to form an image with reasonable contrast; (3) Turning on the fill light device for a long time may attract fish, etc., affecting sonar detection and imaging of the visible light imaging device, and causing the sonar point cloud and visible light image content perceived by Tiger Hill to deteriorate, which is not conducive to correct identification.

[0086] S132: Determine whether the underwater light intensity reaches the clear imaging intensity based on the preliminary visible light image; if so, execute S133; if not, execute S134.

[0087] In the disclosed embodiment, whether the underwater light intensity reaches the clear imaging intensity based on the preliminary visible light image can be determined by calibrating the black and white fields of the preliminary visible light image after obtaining the preliminary visible light image, re-determining the color scale of the preliminary visible light image based on the calibrated black and white fields, and analyzing whether the color scale of each channel is smooth and reasonable, so as to ensure that the entire visible light image forms a reasonable hierarchical distribution, thereby facilitating subsequent processing.

[0088] If it is determined after the above processing that the visible light image still cannot form a reasonable layer distribution, it is determined that the underwater light intensity cannot reach the clear imaging light intensity, and the subsequent S134 is executed accordingly. Otherwise, the following S133 is executed.

[0089] S133: Using the preliminary visible light image as the visible light image.

[0090] S134: Control the automatic light-filling device to fill light into the underwater environment, and simultaneously control the visible light imaging system to photograph the underwater environment to obtain a visible light image.

[0091] In a specific implementation, the computing device can control the automatic fill light device to gradually increase the illumination intensity of the automatic fill light device, and at the same time control the visible light imaging system to capture the underwater environment to obtain multiple images (or multiple images), and select the image with better quality (contrast and clarity) as the visible light image. In practical applications, multiple visible light images can be processed at the same time, and the user's subsequent data can be verified multiple times.

[0092] In practical applications, in order to ensure the reliability of sonar detection, avoid the impact of sonar detection on aquatic organisms, and reduce the power consumption of sonar, underwater robots usually detect and form sonar point clouds in areas close to the bottom of the water (for example, 0.5m close to the bottom of the water). In actual situations, the size of suspicious objects can be large or small, and in order to reasonably determine the type of suspicious objects, panoramic imaging of suspicious objects is performed as much as possible. The premise for the visible light imaging system to form panoramic imaging is to have a reasonable distance relative to the object, and its field of view basically covers the entire suspicious object.

[0093] Figure 3 1 is a flow chart of a method for determining adjustment parameters of an underwater robot provided by some embodiments of the present disclosure. Figure 3 As shown, the method S150-S170 shown is to reasonably adjust the underwater and robot so that the visible light imaging system can capture the full picture of the suspicious object as much as possible.

[0094] S150: Determine an estimated size of the suspicious object based on the sonar point cloud, and determine a relative position of the suspicious object and the underwater robot based on the sonar point cloud.

[0095] Determining the estimated size of the suspicious object based on the sonar point cloud means determining the length and width characteristics of the suspicious object based on the range of the shape characteristics and the accuracy characteristics of the sonar point cloud in the process of determining the shape characteristics of the suspicious object using the previous method.

[0096] Determining the relative position of the suspicious object and the underwater robot based on the sonar point cloud is to determine the position of the suspicious object and the underwater robot's reference coordinate system based on the sonar point cloud. The aforementioned reference coordinate system can be a coordinate system based on the sonar system, a coordinate system based on the robot center, or a coordinate system based on the visible light imaging system.

[0097] In a specific implementation, the key feature points (such as the point cloud cluster center and boundary points) representing the suspicious object in the sonar point cloud can be used as a comparison benchmark to determine the relative position of the suspicious object and the underwater robot. The relative position mentioned here is the relative position in the three-dimensional coordinate system, that is, the relative position has three coordinate dimensions.

[0098] S160: Based on the estimated size and relative position, determine the posture adjustment parameters of the underwater robot and / or determine the adjustment internal parameters of the shooting lens in the visible light imaging system.

[0099] In order to capture the entire suspicious object, according to the above analysis, the position of the underwater robot needs to be adjusted so that the camera lens of the visible light imaging system faces the suspicious object at a reasonable position, and the focal length is adjusted so that each part of the suspicious object is imaged in the visible light imaging system. Based on the above purpose, the computing device in the embodiment of the present disclosure determines the posture adjustment parameters of the underwater robot and the adjustment internal parameters of the camera lens according to the estimated size and relative position.

[0100] The posture adjustment parameters are adjustment parameters used to adjust the position and posture of the underwater robot in the world coordinate system, which include translation parameters and rotation parameters. The translation parameters characterize how the underwater robot performs translational motion underwater. The translation parameters are parameters including three coordinate dimensions (if there is no translation in a certain dimension, the parameter of the corresponding dimension is 0). The rotation parameters characterize the parameters of the underwater robot's rotation around the reference coordinate system. The goal of the underwater robot to adjust according to the posture adjustment parameters is to enable the camera lens to face the suspicious object directly and to be at a reasonable distance from the suspicious object (this distance, in conjunction with the adjustment internal parameters of the camera lens, can form a full-view image of the suspicious object and form an area around the suspicious object to better distinguish the suspicious object).

[0101] Adjusting the internal parameters is a parameter used to adjust the focal length of the camera lens. It is used to adjust the field of view of the visible light imaging system and cooperates with the above-mentioned posture adjustment parameters to form a reasonable result.

[0102] S170: Control the underwater robot to make adjustments according to the posture adjustment parameters and the internal parameters.

[0103] In a specific implementation, an inertial measurement unit is provided in the underwater robot, which can provide feedback control on the posture adjustment state of the underwater robot to determine whether the posture adjustment parameters are adjusted in place. Similarly, a feedback unit is also provided in the camera lens to provide feedback on whether the camera lens is adjusted in place according to the adjustment internal parameters.

[0104] When executing the aforementioned S150-S170, the aforementioned S130 controls the visible light imaging system installed on the underwater robot to shoot the underwater environment to obtain a visible light image. Specifically, after the underwater robot adjusts its posture position according to the posture adjustment parameters, and the shooting lens is adjusted according to the adjustment internal parameters, the visible light imaging system is controlled to shoot the underwater environment to obtain a visible light image.

[0105] In some specific applications, the camera lens of the visible light imaging system may be fixed on the underwater robot, and its external posture is fixed relative to the underwater robot. In this case, it is only possible to adjust the underwater robot using S170 as described above to achieve the goal of imaging the entire object.

[0106] In some other specific applications, the camera lens of the visible light imaging system is installed on the underwater robot in a multi-degree-of-freedom manner, and it can interact with the underwater robot body. In a specific implementation, the aforementioned activities are mostly rotations relative to a specific rotation axis. In this case, while executing S160 to determine the posture adjustment parameters and adjustment internal parameters of the underwater robot, the adjustment external parameters of the camera lens can also be determined. The adjustment external parameters of the camera lens are parameters used to adjust the posture of the camera lens relative to the underwater robot body. In a specific implementation, the adjustment external parameters are mostly rotational external parameters. When the adjustment external parameters are determined, the aforementioned S130 controls the visible light imaging system installed on the underwater robot to shoot the underwater environment to obtain a visible light image, specifically: after the underwater robot adjusts the posture position according to the posture adjustment parameters, and the camera lens is adjusted according to the adjustment internal parameters, and after the camera lens is adjusted according to the adjustment external parameters, the visible light imaging system is controlled to shoot the underwater environment to obtain a visible light image.

[0107] In some embodiments, the aforementioned step S130 of processing the visible light image to determine the appearance features of the suspicious object may include the following steps S130A-S130D.

[0108] S130A: Determine a posture adjustment feature of the shooting lens based on the posture adjustment parameter and the adjustment extrinsic parameter.

[0109] In a specific implementation, the computing device may integrate the posture adjustment parameters and the adjustment external parameters to determine the posture adjustment characteristics of the shooting lens. For example, when the posture adjustment parameters and the adjustment external parameters both include translation parameters and rotation parameters, and the translation parameters are represented by translation matrices and the rotation parameters are represented by rotation matrices, the translation matrices may be added and the rotation matrices may be multiplied to obtain new translation matrices and rotation matrices, and the aforementioned matrices may represent the posture adjustment characteristics of the shooting lens.

[0110] S130B: Determine the imaging viewing angle scaling feature of the shooting lens based on the adjusted internal parameters.

[0111] Adjusting the internal parameter is used to change the focal length of the shooting lens, which also changes the imaging area of ​​the imaging region in the visible light imaging system. In a specific implementation, the imaging angle scaling feature of the shooting lens can be determined based on the correspondence between the adjustment internal parameter, the internal parameter before adjustment, and the imaging area. In simple terms, the aforementioned imaging angle scaling feature reflects the change ratio of the pixel area formed by a unit size at a specific imaging distance.

[0112] S130C: Determine the local image of the suspicious object in the visible light image based on the posture adjustment feature and the imaging perspective scaling feature, the position of the suspicious object in the sonar point cloud, and the coordinate system conversion relationship between the sonar system and the visible light imaging system.

[0113] In a specific implementation, after obtaining the characteristics of adjusting the posture and imaging perspective scaling, the image change of a specific area in the sonar coordinate system in the visible light imaging system can be determined (this is a virtual representation and may not be truly imaged in actual situations). After determining the position of the suspicious object in the sonar point cloud, the image change of the suspicious object in the visible light imaging system can be determined. Combining the coordinate system conversion relationship between the sonar system and the visible light imaging system, the target pixel area of ​​the suspicious object in the visible light image can be determined. The target pixel area is the pixel area in the visible light image that displays the suspicious object.

[0114] S130D: Process local images to determine the appearance characteristics of suspicious objects.

[0115] After obtaining the partial image, the aforementioned processing method can be used to determine the appearance characteristics of the suspicious object.

[0116] In specific implementations, the underwater robot power device may stir up the underwater sludge, or aquatic organisms may stir up the underwater sludge, causing the captured visible light image to be a muddy water image. In this case, the muddy water obscures the suspicious object, making it impossible to form an image of the suspicious object, and thus impossible to determine the image of the suspicious object.

[0117] Correspondingly, when executing the aforementioned S131D, it is specifically S130D1-S130D2.

[0118] S130D1: Perform preliminary recognition on the visible light image to determine whether the visible light image is an underwater image without muddy water features; if so, execute S130D2.

[0119] In a specific implementation, the computing device can determine whether the visible light image is an underwater image with non-muddy water characteristics by analyzing the sludge particle size state characteristics and the sludge particle size distribution characteristics of the water body in the visible light image.

[0120] S130D2: Process the visible light image to determine the appearance characteristics of the suspicious object.

[0121] Obviously, the use of S130D1 and S130D2 can eliminate invalid data processing and thus eliminate erroneous judgments on the types of suspicious objects.

[0122] In a specific implementation, after obtaining the visible light image, the computing device can also store the visible light image in a memory for subsequent data tracing. In addition, the computing device can also output the visible light image through a real device, and relevant technical personnel can determine whether the suspicious object is an underwater suspicious facility that affects water quality.

[0123] The disclosed embodiment also provides a device for identifying suspicious underwater facilities that affect water quality. Figure 4 Schematic diagram of the structure of the device for identifying suspicious underwater facilities that affect water quality provided by the embodiment of the present disclosure. Figure 4 As shown, the underwater suspicious facility identification device 400 that affects water quality provided by the embodiment of the present disclosure includes a point cloud processing unit 401, a judgment unit 402, a control unit 403 and an image processing unit 404.

[0124] The point cloud processing unit 401 is used to obtain the sonar point cloud obtained by the sonar system installed on the underwater robot to detect the underwater environment, and process the sonar point cloud to determine the shape characteristics of the underwater object.

[0125] The judging unit 402 is used to judge whether the underwater object includes a suspicious object based on the appearance characteristics.

[0126] The control unit 403 is used to control the visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image when it is determined that the underwater object includes a suspicious object.

[0127] The image processing unit 404 is used to process the visible light image to determine the appearance characteristics of the suspicious object.

[0128] The judgment unit 402 is also used to identify whether the suspicious object is an underwater suspicious facility that affects water quality based on appearance features.

[0129] In some embodiments, the control unit 403 controls the visible light imaging system to capture the underwater environment to obtain a preliminary visible light image.

[0130] The judgment unit 402 judges whether the underwater light intensity reaches the clear imaging intensity based on the preliminary visible light image; when the underwater light intensity reaches the clear imaging intensity, the preliminary visible light image is used as the visible light image; and, when the underwater light intensity does not reach the clear imaging intensity, the control unit 403 controls the automatic fill light device to fill light to the underwater environment, and simultaneously controls the visible light imaging system to shoot the underwater environment to obtain a visible light image.

[0131] In some embodiments, when it is determined that the underwater object includes a suspicious object, a posture adjustment unit is also included. The posture adjustment unit determines the estimated size of the suspicious object based on the sonar point cloud, and determines the relative position of the suspicious object and the underwater robot based on the sonar point cloud; based on the estimated size and relative position, the posture adjustment parameters of the underwater robot and / or the adjustment internal parameters of the shooting lens in the visible light imaging system are determined. The control unit 403 controls the underwater robot to make corresponding adjustments according to the posture adjustment parameters and the adjustment internal parameters, and when the underwater robot adjusts the posture position according to the posture adjustment parameters and the shooting lens is adjusted according to the adjustment internal parameters, the visible light imaging system is controlled to shoot the underwater environment to obtain a visible light image.

[0132] In some embodiments, the shooting lens is an adjustable lens relative to the underwater robot body. The posture adjustment unit is also used to determine the posture adjustment parameters of the underwater robot and / or determine the adjustment internal parameters of the shooting lens based on the estimated size and relative position, while determining the adjustment external parameters of the shooting lens; the corresponding control unit is also used to adjust the shooting camera according to the adjustment external parameters. When the underwater robot adjusts the posture position according to the posture adjustment parameters, and the shooting lens is adjusted according to the adjustment internal parameters, and the shooting lens is adjusted according to the adjustment external parameters, the control unit 403 controls the visible light imaging system to shoot the underwater environment to obtain a visible light image.

[0133] In some embodiments, the posture adjustment unit determines the posture adjustment feature of the shooting lens based on the posture adjustment parameters and the adjustment external parameters; and determines the imaging angle scaling feature of the shooting lens based on the adjustment internal parameters. The corresponding image determines the local image of the suspicious object in the visible light image based on the posture adjustment feature and the imaging angle scaling feature, the position of the suspicious object in the sonar point cloud, the relative position, and the coordinate system transformation relationship between the sonar system and the visible light imaging system, and processes the local image to determine the appearance characteristics of the suspicious object.

[0134] In some embodiments, the image processing unit 404 performs preliminary identification on the visible light image to determine whether the visible light image is an underwater image with non-muddy water characteristics; if it is determined that the visible light image is an underwater image with non-muddy water characteristics, the visible light image is processed to determine the appearance characteristics of the suspicious object.

[0135] In some embodiments, the determination unit 402 identifies whether the suspicious object is an underwater suspicious facility that affects water quality based on shape features and appearance features.

[0136] In some embodiments, the judgment unit 402 obtains a first confidence probability when the appearance feature is a feature of a suspicious underwater facility that affects water quality; obtains a second confidence probability when the appearance feature is a feature of a suspicious underwater facility that affects water quality; then, based on the first confidence probability and the second confidence probability, determines an exclusion probability that the suspicious object is not a suspicious underwater facility that affects water quality; finally, when the exclusion probability is less than a set probability, determines that the suspicious object is a suspicious underwater facility that affects water quality.

[0137] In some embodiments, the control unit 403 is further configured to control the storage of the visible light image and / or control the output of the visible light image through a display device.

[0138] The disclosed embodiment also provides a system for identifying suspicious underwater facilities that affect water quality. Figure 5 Schematic diagram of the structure of the underwater suspicious facility identification system that affects water quality provided by the embodiment of the present disclosure. Figure 5 As mentioned above, the underwater suspicious facility identification system 500 that affects water quality includes a computing device 501 and an underwater robot 502; the underwater robot 502 includes a robot body 5021, and a sonar system 5022 and a visible light imaging system 5023 arranged on the robot body 5021; the computing device 501 is used to determine whether the underwater object has an underwater suspicious facility that affects water quality based on the sonar point cloud obtained by detecting the bottom of the water by the sonar system 5022, according to the previous method for identifying underwater suspicious facilities that affect water quality.

[0139] In some embodiments, in order to realize the rapid return of the data collected by the underwater robot, the underwater robot may also include an image communication module, which is connected to the robot body 5021, the sonar system 5022 and the visible light imaging system 5023 for sending and receiving various information. In addition, the underwater robot may also be equipped with a manual remote control device or a global positioning system to realize long-distance remote control or autonomous cruising according to a preset track.

[0140] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying suspicious underwater facilities that affect water quality, characterized in that: include: Acquire a sonar point cloud obtained by a sonar system installed on the underwater robot to detect the underwater environment, and process the sonar point cloud to determine the shape characteristics of the underwater object; Determining whether the underwater object includes a suspicious object based on the appearance features; In the case where it is determined that the underwater object includes a suspicious object, controlling a visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image, and processing the visible light image to determine the appearance characteristics of the suspicious object; Based on the appearance features, it is identified whether the suspicious object is a suspicious underwater facility that affects water quality.

2. The method according to claim 1, characterized in that The control of the visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image also includes: Controlling the visible light imaging system to photograph the underwater environment to obtain a preliminary visible light image; Determining whether the underwater light intensity reaches the clear imaging light intensity based on the preliminary visible light image; When the underwater light intensity reaches the clear imaging light intensity, taking the preliminary visible light image as the visible light image; and, When the underwater light intensity does not reach the clear imaging light intensity, the automatic light filling device is controlled to fill light into the underwater environment, and at the same time, the visible light imaging system is controlled to photograph the underwater environment to obtain the visible light image.

3. The method according to claim 1, characterized in that In the case where it is determined that the underwater object includes a suspicious object, the method further includes: determining an estimated size of the suspicious object based on the sonar point cloud, and determining a relative position of the suspicious object and the underwater robot based on the sonar point cloud; Before controlling the visible light imaging system installed on the underwater robot to photograph the underwater environment to obtain a visible light image, the method further includes: determining a posture adjustment parameter of the underwater robot and / or determining an adjustment internal parameter of a shooting lens in the visible light imaging system based on the estimated size and the relative position; The method of controlling a visible light imaging system installed on an underwater robot to photograph an underwater environment to obtain a visible light image comprises: when the underwater robot adjusts its posture position according to the posture adjustment parameters and the shooting lens is adjusted according to the adjustment internal parameters, controlling the visible light imaging system to photograph the underwater environment to obtain the visible light image.

4. The method according to claim 3, characterized in that The shooting lens is a lens that is adjustable relative to the underwater robot body, and the method further includes: Based on the estimated size and the relative position, determine the posture adjustment parameters of the underwater robot and / or determine the adjustment internal parameters of the shooting lens, and determine the adjustment external parameters of the shooting lens; After the underwater robot adjusts its position according to the position adjustment parameter, and the shooting lens is adjusted according to the adjustment internal parameter, controlling the visible light imaging system to shoot the underwater environment to obtain the visible light image, including: When the underwater robot adjusts its posture position according to the posture adjustment parameters, and the shooting lens is adjusted according to the adjustment internal parameters, and the shooting lens is adjusted according to the adjustment external parameters, the visible light imaging system is controlled to shoot the underwater environment to obtain the visible light image.

5. The method according to claim 4, characterized in that The processing of the visible light image to determine the appearance characteristics of the suspicious object includes: Determining a posture adjustment feature of the shooting lens based on the posture adjustment parameter and the adjustment extrinsic parameter; Determining the imaging viewing angle scaling feature of the shooting lens based on the adjustment internal parameter; Determine a local image of the suspicious object in the visible light image based on the posture adjustment feature and the imaging view scaling feature, the position of the suspicious object in the sonar point cloud, the relative position, and the coordinate system transformation relationship between the sonar system and the visible light imaging system; The partial image is processed to determine the appearance features of the suspicious object.

6. The method according to any one of claims 1 to 5, characterized in that: The processing of the visible light image to determine the appearance characteristics of the suspicious object includes: Performing preliminary recognition on the visible light image to determine whether the visible light image is an underwater image with non-muddy water features; In the case where it is determined that the visible light image is an underwater image with non-muddy water characteristics, the visible light image is processed to determine the appearance characteristics of the suspicious object.

7. The method according to any one of claims 1 to 5, characterized in that: The step of identifying whether the suspicious object is an underwater suspicious facility that affects water quality based on the appearance feature includes: Based on the shape features and the appearance features, it is identified whether the suspicious object is a suspicious underwater facility that affects water quality.

8. The method according to claim 7, characterized in that The step of identifying whether the suspicious object is a suspicious underwater facility that affects water quality based on the shape feature and the appearance feature includes: In the case where the appearance feature is a feature of a suspicious underwater facility that affects water quality, obtaining a first certainty probability; In the case where the appearance feature is a feature of a suspicious underwater facility that affects water quality, obtaining a second certainty probability; Determining, based on the first certainty probability and the second certainty probability, a probability of eliminating the possibility that the suspicious object is not a suspicious underwater facility that affects water quality; When the exclusion probability is less than the set probability, the suspicious object is determined to be a suspicious underwater facility that affects water quality.

9. The method according to any one of claims 1 to 4, characterized in that: After processing the visible light image to determine the appearance characteristics of the suspicious object, the method further includes: The visible light image is stored, and / or the visible light image is output through a display device.

10. A system for identifying suspicious underwater facilities that affect water quality, characterized in that: It includes a computing device and an underwater robot; the underwater robot includes a robot body, and a sonar system and a visible light imaging system arranged on the robot body; The computing device is used to determine whether the underwater object is an underwater suspicious facility that affects the water quality based on the sonar point cloud obtained by detecting the bottom of the water by the sonar system, according to the method for identifying underwater suspicious facilities that affect the water quality as described in any one of claims 1 to 9.

Images