Underwater life body quantity monitoring device based on computer vision

By integrating AI visual recognition cameras and underwater laser scanning imagers on the moving boxes underwater, combined with wireless communication and feature extraction algorithms, the problem of the existing underwater life quantity monitoring equipment requiring long-term deployment and manual carrying is solved. Convenient and efficient monitoring of the number of underwater living organisms has been achieved.

CN119006996AInactive Publication Date: 2024-11-22HANNING REMOTE SENSING TECH RES INST (NANJING) CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411490752.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-24
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing underwater life quantity monitoring equipment usually needs to be deployed in specific locations for long-term continuous monitoring, and portable equipment needs to be carried manually, resulting in high work intensity and is not conducive to long-term monitoring, especially in the area of ​​underwater life counting. There are difficulties.

Method used

A computer vision-based underwater life quantity monitoring device is designed, using a box with a thruster to move, combining an AI vision recognition camera and an underwater laser scanning imager to transmit image data to the PC terminal through wireless communication. The process is carried out, and the feature extraction and matching algorithms are used to compare two-dimensional and three-dimensional image data, and the number of underwater life forms is counted.

Benefits of technology

Convenient underwater movement monitoring is achieved, the accuracy and stability of monitoring is improved, the rotation structure is simplified, the device volume is reduced, the working intensity is reduced, and the counting efficiency of the number of underwater living organisms is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006996A_ABST
    Figure CN119006996A_ABST
Patent Text Reader

Abstract

The invention discloses an underwater life body quantity monitoring device based on computer vision, which belongs to the field of vision monitoring and comprises a box body, a transparent waterproof cover is arranged at the top end of the box body, and a vision sensor is arranged in the transparent waterproof cover and connected with a turntable through an angle adjusting assembly. The bottom end of the box body is provided with an underwater laser scanning imager, the underwater laser scanning imager is connected with the rotating disc through a rotating assembly in the box body, the box body is internally provided with a floating and sinking structure used for sinking, and the visual sensor and the underwater laser scanning imager are connected with a PC terminal through a wireless communication assembly. According to the underwater life body number monitoring device based on the computer vision, the underwater life body is monitored through the box body moving underwater, the AI visual recognition camera is used for collecting underwater life body visual image data, the underwater laser scanning imager is used for collecting underwater life body three-dimensional image data, and the underwater life body number monitoring device is used for monitoring the underwater life body number. And processing the two image data to obtain the number of underwater life bodies.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of visual monitoring, and in particular to a device for monitoring the number of underwater life forms based on computer vision. Background Art

[0002] Underwater life monitoring usually refers to the process of detecting, tracking and studying organisms in the underwater environment using specific technologies and equipment. This type of monitoring is very important in areas such as environmental protection, scientific research, fishery management and water safety. Modern underwater life monitoring technologies include: Sonar technology: Sonar (Sound Navigation And Ranging) uses the properties of sound waves propagating underwater to locate and identify objects, including living things. Doppler sonar and side-scan sonar can provide detailed maps of the seafloor topography and underwater life distribution.

[0003] Underwater cameras and vision systems: High-definition cameras are used to directly capture videos or photos of underwater creatures. They can be combined with LED lighting and a variety of optical filters to adapt to shooting needs at different depths and light conditions.

[0004] Infrared thermal imaging: Although not as widely used underwater as in the air, infrared thermal imaging technology can help identify organisms with temperature differences under certain specific conditions.

[0005] Bioacoustic tags: Small electronic tags can be attached to animals to record their location, depth and other behavioral data, and then transmit the data back through a network of acoustic receivers.

[0006] Chemical sensors: These are used to detect the presence of specific chemicals in water, such as biomarkers, which can help indirectly infer the type and amount of organisms.

[0007] Environmental DNA (eDNA): By collecting and analyzing DNA in water samples, it is possible to identify species present in a body of water, even if they have not been directly observed.

[0008] Remote sensing techniques: Although primarily used above the water surface, satellite and aerial remote sensing can also provide information on ocean surface conditions, indirectly reflecting the state of underwater ecosystems.

[0009] Existing underwater life monitoring equipment is generally deployed at specific underwater locations for a long time for continuous monitoring to collect information on the number of life forms. Portable equipment is usually carried manually, and counting underwater life forms in the area requires high labor intensity and long working hours, which is not conducive to long-term monitoring. Summary of the invention

[0010] The purpose of the present invention is to provide an underwater life form quantity monitoring device based on computer vision, which facilitates the monitoring and detection of life forms in the area through a box that can move underwater, collects visual image data of underwater life forms through an AI visual recognition camera, and collects three-dimensional image data of underwater life forms through an underwater laser scanning imager, and the number of underwater life forms is obtained after processing the image data of the two.

[0011] To achieve the above-mentioned purpose, the present invention provides an underwater life quantity monitoring device based on computer vision, including a box body, a transparent waterproof cover is arranged on the top of the box body, a visual sensor for monitoring underwater life bodies is arranged in the transparent waterproof cover, the visual sensor is connected to the turntable through an angle adjustment component, an underwater laser scanning imager for monitoring underwater life bodies is arranged at the bottom of the box body, the underwater laser scanner is connected to the turntable through a rotating component inside the box body, a floating and sinking structure for sinking is arranged in the box body, the visual sensor and the underwater laser scanning imager are connected to a PC through a wireless communication component, and an illumination lamp is arranged on the box body.

[0012] Preferably, horizontal propellers are symmetrically arranged on both sides of the box body, and the box body is moved forward and turned by the horizontal propellers. Vertical propellers are arranged at the four corners of the box body, and the box body is moved upward by the vertical propellers.

[0013] Preferably, a support frame is provided at the bottom of the box body, and the support frame is used to support the box body when it falls to the ground. The support frame includes a connecting rod, and the bottom connecting rod of the box body is connected to the support plate, and a plurality of strip holes are provided on the support plate.

[0014] Preferably, the visual sensor adopts an AI visual recognition camera.

[0015] Preferably, the angle adjustment assembly includes two fixed plates arranged on a turntable, the turntable is rotatably connected to the box, a rotating rod is rotatably arranged on the fixed plate, the rotating rod is connected to the bottom end of the visual sensor, the rotating rod passes through the fixed plates and limited blocks are arranged at both ends, and one end of the rotating rod passes through the fixed plate and is connected to the output end of the adjustment motor through a worm gear structure.

[0016] Preferably, the rotating assembly includes a rotating shaft, which is rotatably sealed with the box body, and the bottom end of the rotating shaft passes through the bottom of the box body and is connected to the base, and the underwater laser scanning imager is arranged on the base.

[0017] Preferably, the top end of the rotating shaft is connected to the bottom end of the turntable, a central gear is provided on the rotating shaft, and the central gear is connected to the rotating structure.

[0018] Preferably, the rotating structure includes a rotating motor, which is symmetrically arranged in the housing, and the output shaft of the rotating motor is connected to the central gear through a circular gear.

[0019] Preferably, the floating and sinking structure includes a water tank symmetrically arranged in the box body, the water tank is connected to a water pipe through a water pump, and the pipe head of the water pipe is arranged at the bottom of the box body.

[0020] The present invention also provides a method for monitoring the number of underwater life forms based on computer vision, comprising the following steps: Step 1: collect some visual image data of underwater life forms through the AI ​​visual recognition camera of the underwater life form quantity monitoring device, and transmit the image data to the PC end; Step 2: Collecting a number of three-dimensional image data of underwater life forms through an underwater laser scanning imager of an underwater life form quantity monitoring device, and transmitting the image data to a PC; Step 3: Preprocess and reduce noise of the visual image data, compare the image database of life forms in the target waters with the processed visual image data through feature extraction and matching algorithms, segment and extract the image data that meets the target waters, and obtain the preliminary number of underwater life forms; Step 4: pre-process and reduce noise on the 3D image data, compare the image database of life forms in the target waters with the processed 3D image data through feature extraction and matching algorithms, segment and extract the image data that meets the target waters, and obtain a preliminary number of underwater life forms; Step 5: Take the average of the number of underwater life forms obtained through the visual image data and the number of underwater life forms obtained through the three-dimensional image data to obtain the final number of underwater life forms.

[0021] Preferably, the target water area life image database includes life visual image data and three-dimensional image data; The steps of feature extraction and matching algorithm in step 4 include: S11, detect key points: find the positions of corner points and edge key points in the preprocessed visual image; S12, descriptor calculation: Descriptors are generated around the detected key points to identify the same features under different conditions; S13, normalization: normalize the grayscale of the identified image features; S14, similarity measure: calculate the similarity of key point descriptors in visual images; S15, nearest neighbor search: finding the point in one visual image that has the most similar descriptor in another image; S16, matching verification: using the RANSAC random sampling consensus algorithm to remove false matches and retain consistent feature matches; The steps of feature extraction and matching algorithm in step 5 include: S21, surface normal estimation: estimating the surface normal of each feature point of the three-dimensional image data; S22, feature point detection: detecting corner points in the three-dimensional image data; S23, feature description: generate a descriptor for each feature point through the FPFH algorithm; S24, similarity measurement: comparing feature descriptors in point clouds of different three-dimensional image data to determine corresponding feature points; S25, nearest neighbor search: find the best match for each feature point in the source point cloud in the target point cloud; S26, Matching Optimization: Use the RANSAC random sampling consensus algorithm to remove false matches and estimate the rigid body transformation between the two sets of point clouds.

[0022] Therefore, the present invention adopts the above-mentioned underwater life form quantity monitoring device based on computer vision, which has the following benefits: (1) The present invention uses a box with a propeller to move the box underwater, which is convenient for the box to detect underwater life in the area. After the box falls to the bottom of the area through the support frame at the bottom, the underwater laser scanning imager at the bottom of the box will not be squeezed; (2) The present invention drives the rotating shaft to rotate through a rotating motor and a gear transmission structure. The rotation of the rotating shaft synchronously drives the visual sensor and the underwater laser scanning imager to rotate. Both can be driven to rotate by a single rotating shaft, which not only improves the stability during the rotation process, but also simplifies the rotating structure and occupies less space in the box, thereby reducing the size of the entire device and improving ease of use; (3) The present invention collects visual two-dimensional images and three-dimensional point cloud images through visual sensors and underwater laser scanning imagers respectively, and completes the counting of underwater life forms by comparing and extracting the image data in the target water area. The counting is achieved by combining two-dimensional and three-dimensional methods to improve accuracy.

[0023] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A three-dimensional diagram of an underwater life quantity monitoring device according to an embodiment of the present invention; Figure 2 It is a front view of the underwater life quantity monitoring device according to an embodiment of the present invention; Figure 3 for Figure 2 A magnified schematic diagram of point A; Figure 4 It is a side view of the underwater life quantity monitoring device according to an embodiment of the present invention; Figure 5 Schematic diagram of the water tank position according to an embodiment of the present invention.

[0025] Reference numerals 1. Box body; 2. Transparent waterproof cover; 3. Visual sensor; 4. Underwater laser scanning imager; 5. Illumination lamp; 6. Horizontal thruster; 7. Vertical thruster; 8. Support frame; 9. Connecting rod; 10. Support plate; 11. Strip hole; 12. Turntable; 13. Fixed plate; 14. Rotating rod; 15. Limit block; 16. Adjusting motor; 17. Worm gear structure; 18. Rotating shaft; 19. Rotating motor; 20. Circular gear; 21. Center gear; 22. Water tank; 23. Water pump; 24. Water pipe. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0027] It should be noted that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0028] Like reference numerals and letters denote similar items in the following drawings, and thus, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, or are directions or positional relationships in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0030] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "setting", "installation" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0031] like Figure 1 As shown, the underwater life quantity monitoring device based on computer vision described in the present invention includes a box body 1, and horizontal thrusters 6 are symmetrically arranged on both sides of the box body 1. The box body 1 moves forward and turns through the horizontal thrusters 6. Vertical thrusters 7 are arranged at the four corners of the box body 1, and the box body 1 moves upward through the vertical thrusters 7. A support frame 8 is arranged at the bottom of the box body 1, and the support frame 8 is used to support the box body 1 when it falls to the ground. The support frame 8 includes a connecting rod 9. The bottom connecting rod 9 of the box body 1 is connected to a support plate 10. The support plate 10 is provided with a plurality of strip holes 11 for reducing water flow resistance. An illuminating lamp 5 is arranged on the box body 1, and clearer visual image data can be collected through the illuminating lamp 5.

[0032] like Figure 2 , Figure 3 , Figure 4 As shown, a transparent waterproof cover 2 is provided at the top of the box 1, and a visual sensor 3 for monitoring underwater life is provided inside the transparent waterproof cover 2. The visual sensor 3 uses an AI visual recognition camera, which can use AI to identify underwater objects and eliminate some image data of non-living objects, thereby reducing the complexity of subsequent image data processing and improving efficiency. An underwater laser scanning imager 4 for monitoring underwater life is provided at the bottom of the box 1. The underwater laser scanning imager 4 can generate a three-dimensional image of underwater objects, and count them together with the visual two-dimensional image data to improve the accuracy of counting. The visual sensor 3 and the underwater laser scanning imager 4 are connected to the PC through a wireless communication component, which is provided on the box 1. The wireless communication component adopts an existing structure to transmit the image data to the PC for processing.

[0033] The visual sensor 3 is connected to the turntable 12 through an angle adjustment component. The angle adjustment component includes two fixed plates 13 arranged on the turntable 12, the turntable 12 is rotatably connected to the box 1, a rotating rod 14 is rotatably arranged on the fixed plate 13, the rotating rod 14 is connected to the bottom end of the visual sensor 3, the rotating rod 14 passes through the fixed plate 13 and is provided with a limit block 15 at both ends, and the rotating rod 14 passes through one end of the fixed plate 13 and is connected to the output end of the adjustment motor 16 through a worm gear structure 17. The worm gear structure 17 includes a worm wheel and a worm, the worm wheel is arranged at one end of the rotating rod 14, one end of the worm is connected to the output shaft of the adjustment motor 16, and the worm is connected to the worm wheel. The adjustment motor 16 drives the rotating rod 14 to rotate, thereby driving the visual sensor 3 to move, and realizing the 0-180 degree pitch angle adjustment of the visual sensor 3. The limit block 15 can limit the left and right sliding of the rotating rod 14, and improve the stability of the visual sensor 3 when rotating.

[0034] The underwater laser scanner is connected to the turntable 12 through a rotating assembly inside the housing 1. The rotating assembly includes a rotating shaft 18, which is rotatably sealed with the housing 1. The bottom end of the rotating shaft 18 passes through the bottom of the housing 1 and is connected to the base, and the underwater laser scanning imager 4 is arranged on the base. The top of the rotating shaft 18 is connected to the bottom end of the turntable 12, and a central gear 21 is arranged on the rotating shaft 18, and the central gear 21 is connected to the rotating structure. The rotating structure includes a rotating motor 19, which is symmetrically arranged in the housing 1, and the output shaft of the rotating motor 19 is connected to the central gear 21 through a circular gear 20. The rotating motor 19 drives the circular gear 20 to rotate, and the circular gear 20 drives the central gear 21 to rotate, thereby driving the rotating shaft 18 to rotate, and the rotation of the rotating shaft 18 drives the upper turntable 12 and the underwater laser scanner below to rotate, and the turntable 12 drives the visual sensor 3 to rotate.

[0035] like Figure 2 , Figure 5 As shown, a floating and sinking structure for sinking is provided in the box body 1, and the floating and sinking structure includes a water tank 22 symmetrically arranged in the box body 1, and the water tank 22 is connected to a water pipe 24 through a water pump 23, and the pipe head of the water pipe 24 is arranged at the bottom of the box body 1. The water pump 23 sucks water into the water tank 22 to increase the weight of the box body 1, so that the box body 1 sinks, and the water pump 23 drains water and cooperates with the vertical thruster 7 to make the box body 1 float.

[0036] The present invention provides a method for monitoring the number of underwater life forms based on computer vision, comprising the following steps: Step 1: Collect some visual image data of underwater life forms through the AI ​​visual recognition camera of the underwater life form quantity monitoring device, and transmit the image data to the PC.

[0037] Step 2: Collect a number of three-dimensional image data of underwater life forms through the underwater laser scanning imager 4 of the underwater life form quantity monitoring device, and transmit the image data to the PC end.

[0038] Step 3: Compare and analyze the image data of life forms in the target waters: The target water area life image database includes life visual image data and three-dimensional image data.

[0039] The visual image data is preprocessed to reduce noise, and the image database of target water life is compared with the processed visual image data through feature extraction and matching algorithms. The image data that meets the target water area is segmented and extracted to obtain a preliminary number of underwater life forms.

[0040] Step 4: Preprocess and reduce noise on the three-dimensional image data, compare the target water area life image database with the processed three-dimensional image data through feature extraction and matching algorithms, segment and extract the image data that meets the target water area, and obtain a preliminary number of underwater life forms.

[0041] Step 5: Take the average of the number of underwater life forms obtained through the visual image data and the number of underwater life forms obtained through the three-dimensional image data to obtain the final number of underwater life forms.

[0042] The steps of feature extraction and matching algorithm in step 4 include: S11, detect key points: find the positions of corner points and edge key points in the preprocessed visual image; S12, descriptor calculation: Descriptors are generated around the detected key points to identify the same features under different conditions; S13, normalization: normalize the grayscale of the identified image features; S14, similarity measure: calculate the similarity of key point descriptors in visual images; S15, nearest neighbor search: finding the point in one visual image that has the most similar descriptor in another image; S16, matching verification: RANSAC random sampling consensus algorithm is used to remove false matches and retain consistent feature matches.

[0043] The steps of feature extraction and matching algorithm in step 5 include: S21, surface normal estimation: estimating the surface normal of each feature point of the three-dimensional image data; S22, feature point detection: detecting corner points in the three-dimensional image data; S23, feature description: generate a descriptor for each feature point through the FPFH algorithm; S24, similarity measurement: comparing feature descriptors in point clouds of different three-dimensional image data to determine corresponding feature points; S25, nearest neighbor search: find the best match for each feature point in the source point cloud in the target point cloud; S26, Matching Optimization: Use the RANSAC random sampling consensus algorithm to remove false matches and estimate the rigid body transformation between the two sets of point clouds.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A computer vision-based underwater life form quantity monitoring device, characterized in that: It includes a box body, a transparent waterproof cover is arranged on the top of the box body, a visual sensor for monitoring underwater life forms is arranged in the transparent waterproof cover, the visual sensor is connected to the turntable through an angle adjustment component, an underwater laser scanning imager for monitoring underwater life forms is arranged at the bottom of the box body, the underwater laser scanner is connected to the turntable through a rotating component inside the box body, a floating structure for sinking is arranged in the box body, the visual sensor and the underwater laser scanning imager are connected to the PC through a wireless communication component, and a lighting lamp is arranged on the box body.

2. The underwater life form quantity monitoring device based on computer vision according to claim 1, characterized in that: Horizontal thrusters are symmetrically arranged on both sides of the box body, and the box body is moved forward and turned by the horizontal thrusters. Vertical thrusters are arranged at the four corners of the box body, and the box body is moved upward by the vertical thrusters.

3. The underwater life form quantity monitoring device based on computer vision according to claim 1, characterized in that: A support frame is arranged at the bottom of the box body, and the support frame is used to support the box body when it falls to the ground. The support frame includes a connecting rod, and the bottom connecting rod of the box body is connected to the supporting plate, and a plurality of strip holes are arranged on the supporting plate.

4. The underwater life form quantity monitoring device based on computer vision according to claim 1, characterized in that: The visual sensor uses an AI visual recognition camera.

5. The underwater life form quantity monitoring device based on computer vision according to claim 1, characterized in that: The angle adjustment assembly includes two fixed plates arranged on a turntable, the turntable is rotatably connected to the box body, a rotating rod is rotatably arranged on the fixed plate, the rotating rod is connected to the bottom end of the visual sensor, the rotating rod passes through the fixed plates and limited blocks are arranged at both ends, and one end of the rotating rod passes through the fixed plate and is connected to the output end of the adjustment motor through a worm gear structure.

6. The underwater life form quantity monitoring device based on computer vision according to claim 1, characterized in that: The rotating assembly comprises a rotating shaft, which is rotationally sealedly connected to the box body, and the bottom end of the rotating shaft passes through the bottom of the box body and is connected to the base, and the underwater laser scanning imager is arranged on the base.

7. The underwater life form quantity monitoring device based on computer vision according to claim 6, characterized in that: The top end of the rotating shaft is connected to the bottom end of the rotating disk, a central gear is arranged on the rotating shaft, and the central gear is connected to the rotating structure.

8. The underwater life form quantity monitoring device based on computer vision according to claim 7, characterized in that: The rotating structure comprises a rotating motor, which is symmetrically arranged in a box body, and an output shaft of the rotating motor is connected with a central gear through a circular gear.

9. The underwater life form quantity monitoring device based on computer vision according to claim 1, characterized in that: The floating and sinking structure comprises a water tank symmetrically arranged in the box body, the water tank is connected with a water pipe through a water pump, and the pipe head of the water pipe is arranged at the bottom of the box body.

Citation Information

Patent Citations

  • Underwater monitoring robot

    CN103057681A

  • Counting device and counting system for trapping injurious insects

    CN103914733A

  • Remote-controlled fish-imitating underwater biological detector

    CN105438420A

  • Novel active and passive combined type light and small unmanned aerial vehicle-mounted mine detector

    CN115774267A

  • Intelligent fish identifying and monitoring method and system based on multi-sensor data

    CN117214904A