Underwater visual tactile sensor based on fish sideline bionics and application thereof

Abstract

The invention belongs to the technical field of sensors, and more particularly relates to an underwater visual tactile sensor based on fish lateral line bionics and application thereof, the underwater visual tactile sensor comprises a hollow shell, an inertial measurement unit, an image acquisition unit and a propeller unit; the hollow shell comprises a hemispherical shell body, a middle cylindrical shell body and a base which are connected in sequence, a plurality of mounting holes which are distributed in an array mode are formed in the hemispherical shell body, and the propeller units are fixed in the corresponding mounting holes and form a passive propeller array protruding out of the hemispherical shell body; the inertial measurement unit is arranged on the outer side of the middle cylindrical shell and is used for measuring the yaw angle of the underwater navigation equipment; and the image acquisition unit is hermetically fixed in the center in the base and is used for acquiring motion images of all the propeller units under the action of fluid. According to the invention, the fluid velocity can be accurately estimated, and the position of the underwater visual tactile sensor can be accurately obtained.

Description

Technical Field

[0001] This application belongs to the technical field of sensors. More specifically, it relates to an underwater visual tactile sensor based on fish lateral line bionics and its application. Background Art

[0002] Underwater autonomous systems (such as unmanned underwater vehicles) face severe motion estimation challenges in complex underwater environments, especially in the absence of GPS signals. Accurate proprioceptive motion estimation is crucial for autonomous navigation, positioning, and task execution. However, advanced methods such as seabed markers or pre-deployed acoustic beacons often rely heavily on external environmental features, which are often unavailable in underwater terrains with unclear structures or unexplored areas. In nature, the lateral line system of fish enables them to perceive water flow changes with millimeter-level accuracy in complete darkness. Existing tactile sensors that mimic the fish lateral line system for positioning are usually manufactured using principles such as piezoelectric materials or magnetic induction. For example, Chinese patent document CN107782373A discloses a novel bionic lateral line sensor, which mainly obtains the correspondence between fluid changes and voltage signal changes to obtain the fluid state around the underwater robot. Chinese patent document CN117607985A discloses a device for an underwater bionic lateral line sensor based on a triboelectric nanogenerator, designs the sensor structure to be approximately the same as the fish lateral line structure, and uses the principle of the triboelectric nanogenerator to mimic the perception mechanism of the lateral line. These sensors usually have complex structures, and the detection range of the lateral line sensor based on magnetic induction is limited, and errors are likely to occur in high-speed or complex background flow fields.

[0003] In summary, it is urgent to improve the sensor structure and its application method underwater to solve the above problems and improve the accurate motion estimation of underwater autonomous systems. Summary of the Invention

[0004] Aiming at the defects of the prior art, the purpose of this application is to provide an underwater visual tactile sensor based on fish lateral line bionics and its application, aiming to solve the problem that it is difficult for existing sensors to accurately estimate the body position of underwater autonomous systems.

[0005] To achieve the above object, in a first aspect, the present application provides an underwater visual and tactile sensor based on fish lateral line bionics, including a hollow housing, an inertial measurement unit, an image acquisition unit, and a propeller unit; the hollow housing includes a hemispherical housing, a middle cylindrical housing, and a base connected in sequence. A plurality of mounting holes arranged in an array are formed on the hemispherical housing, and the propeller unit is fixed in the corresponding mounting holes and forms a passive propeller array protruding from the hemispherical housing; the inertial measurement unit is arranged outside the middle cylindrical housing and is used to measure the yaw angle of the underwater visual and tactile sensor; the image acquisition unit is hermetically fixed at the center inside the base and is used to acquire the motion images of all propeller units under the action of fluid.

[0006] Further, the propeller unit includes a propeller, a bearing, and a stepped shaft. The propeller is connected to the bearing through the stepped shaft, and the propeller and the stepped shaft can rotate synchronously; one end of the stepped shaft away from the propeller is limited inside the hemispherical housing and is within the field of view angle of the image acquisition unit.

[0007] Further, an identification label is provided at one end of the stepped shaft inside the hemispherical housing, and the identification label can rotate synchronously with the propeller.

[0008] Further, the propeller is a three-blade propeller; and / or, one of the three-blade propellers is directly opposite to the image acquisition unit.

[0009] Further, an installation cavity is provided inside the base, an installation step is provided at the peripheral edge of the end of the installation cavity, and a waterproof lens is fixed on the installation step to seal the image acquisition unit inside the base; and / or, a waterproof coating is applied at the connection between the middle cylindrical housing and the base.

[0010] Further, a ring-shaped light source is provided at one end of the middle cylindrical housing close to the base, and the light of the ring-shaped light source is directed towards the hemispherical housing; and / or, a water outlet hole is provided on the middle cylindrical housing, and a water inlet hole is provided on the periphery of the installation hole. After the external fluid acts on the propeller unit, it enters the underwater visual and tactile sensor through the corresponding water inlet hole and flows out from the water outlet hole.

[0011] In a second aspect, an application of the underwater visual and tactile sensor as described above is disclosed, including: S1 The image acquisition unit acquires the motion images of each propeller unit in the passive propeller array under the action of water flow; the inertial measurement unit acquires the yaw angle of the underwater visual and tactile sensor; S2 Perform enhancement processing on the motion images acquired in step S1 to obtain enhanced image data; S3 Based on the enhanced image data, obtain the current angular value of the passive propeller array; perform differential processing on the angular values corresponding to the front and rear two frames of images to obtain the angular velocity of the passive propeller array; S4 Input the angular velocity into a convolutional neural network to obtain the velocity estimation values of the underwater visual tactile sensor in the x and y directions; S5 Based on the velocity estimation values and the yaw angle, obtain the position of the underwater visual tactile sensor.

[0012] Further, in step S1, the image acquisition unit acquires the rotational images of the identification tags on each propeller as the motion images.

[0013] Further, in step S2, the method of enhancement processing is to process the perspective change of the motion image through perspective transformation, and transform the motion image into an angle directly facing the image acquisition unit. For the pixel points in the motion image P , use the following formula for pixel position conversion:

[0014] where, P’ is the pixel position after transformation; H is the perspective matrix.

[0015] Further, in step S3, based on the enhanced image data, the method for obtaining the current angular value of the passive propeller array is: S301 Perform threshold segmentation on the enhanced image data to obtain a binary image; S301 Extract the identification tags from the binary image to obtain the rotational angle of the identification tags as the angular value, then the angular velocity of the passive propeller array is expressed as:

[0016] where, ω is the angular velocity, is the rotational angle in the k th frame of image, is the rotational angle in the k+1 th frame of image, f is the shooting frequency of the image acquisition unit.

[0017] It can be understood that the beneficial effects of the above second aspect can refer to the relevant descriptions in the above first aspect, and will not be elaborated here.

[0018] Generally speaking, compared with the prior art, the above technical solutions conceived by this application have the following beneficial effects: (1) One end of the hollow shell of this application is hemispherical. The hemispherical shape can better reduce the water flow resistance of the sensor during movement, and is closer to the bionic configuration of the fish body streamline. A passive propeller array is arranged on the hemispherical shell. The propeller units distributed all over the hemispherical shell can be affected by the fluid in the forward direction of the underwater vehicle, and can also be affected by the water flow perpendicular to the forward direction, ensuring the stability of the speed estimation in all directions of the hemispherical shell and the robustness against noises such as turbulence and bubbles. An inertial measurement unit is provided on the outer shell of the middle cylinder. An image acquisition unit is installed in the base, and it has a large field of view angle, capable of obtaining the motion images of the passive propeller array under the impact of the fluid. Through the above design, the bionic design of the superficial neuromasts (SN) in the fish lateral line is realized, so that the sensor can effectively sense the fluid actions from different directions, realize the visualization of the fluid contact force, enabling this application to have a high accuracy in fluid speed estimation and further improving the position estimation accuracy of application devices such as underwater autonomous systems.

[0019] (2) This application can capture the deformation and texture changes of the contacting object through the optical system, provide accurate tactile information, and does not require complex embedded electronic components. It has a simple and lightweight structure and can be applied to unmanned autonomous vehicles of various sizes. Compared with the existing fish lateral line bionic sensors, it makes up for the problems of their complex structure and low resolution, and also reduces the design and maintenance costs.

[0020] (3) By setting an additional light source, this application improves the problem of poor measurement results when the sensor faces light attenuation underwater. By designing a water inlet hole around the installation hole and a water outlet hole on the middle shell, this application avoids the formation of turbulence in the inner cavity of the underwater visual tactile sensor during the movement of the underwater visual tactile sensor, and can also make the propeller unit rotate more smoothly. The three-blade propeller used in this application can better visualize the water flow, be sensitive enough to the water flow, and form a passive propeller array with a close arrangement but without interference with each other.

[0021] (4) The underwater visual tactile sensor based on fish lateral line bionics in this application needs to effectively establish the correlation between water flow disturbance and the movement of the underwater vehicle body, and realize the underwater body movement estimation without environmental dependence. The sensor application method of this application first preprocesses the original motion image sequence of the collected propeller array, extracts the instantaneous angular velocity of each propeller unit at the current moment, and then uses it as the input value of the neural network model, thus avoiding a large amount of data redundancy and reducing the robustness of the sensor in complex environments such as sunlight direct irradiation and water bubbles and floating impurities. Description of the Drawings

[0022] Figure 1 is an exploded structural schematic diagram of the underwater visual tactile sensor based on fish lateral line bionics provided in Embodiment 1 of this application; Figure 2 It is a schematic diagram of the overall structure of the underwater visual and tactile sensor based on fish lateral line bionics provided in Embodiment 1 of the present application; Figure 3 It is a schematic diagram of the partial structure of the underwater visual and tactile sensor based on fish lateral line bionics provided in Embodiment 1 of the present application; Figure 4 It is an exploded structure schematic diagram of the propeller unit provided in Embodiment 1 of the present application; Figure 5 It is a schematic flow diagram of the application method of the underwater visual and tactile sensor based on fish lateral line bionics provided in Embodiment 2 of the present application; Figure 6 It is a schematic diagram of the estimated result of the body position of the underwater visual and tactile sensor based on fish lateral line bionics provided in Embodiment 2 of the present application.

[0023] In all the drawings, the same reference numerals are used to represent the same elements or structures, where: 1 - Hemispherical housing, 2 - Middle cylindrical housing, 21 - Water outlet hole, 3 - Inertial measurement unit, 4 - Waterproof lens, 5 - Ring-shaped light source, 6 - Sealing waterproof ring, 7 - Image acquisition unit, 8 - Base, 81 - Installation cavity, 82 - Installation step, 9 - Propeller, 10 - Bearing, 11 - Step shaft, 111 - Identification label, 12 - Water inlet hole. Detailed implementation manners

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0025] The term "and / or" in this article is a relationship describing associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The symbol " / " in this article represents an "or" relationship between associated objects. For example, A / B represents A or B.

[0026] The terms "first" and "second" in the description and claims of this article are used to distinguish different objects, rather than to describe a specific order of the objects. For example, the first response message and the second response message are used to distinguish different response messages, rather than to describe the specific order of the response messages.

[0027] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0028] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" refers to two or more. For example, a plurality of processing units refers to two or more processing units, etc.; a plurality of elements refers to two or more elements, etc.

[0029] The embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0030] Embodiment 1 This embodiment provides an underwater visual tactile sensor based on fish lateral line bionics, as Figure 1 and Figure 2 shown. The underwater visual tactile sensor includes a hollow housing, an inertial measurement unit 3, an image acquisition unit 7, and a propeller unit; the hollow housing includes a hemispherical shell 1, a middle cylindrical shell 2, and a base 8 that are connected in sequence. A plurality of mounting holes arranged in an array are formed on the hemispherical shell 1. The propeller units are fixed in the corresponding mounting holes and form a passive propeller array protruding from the hemispherical shell 1; the inertial measurement unit 3 is arranged outside the middle cylindrical shell 2 and is used to measure the yaw angle of the underwater visual tactile sensor (i.e., the yaw angle of the underwater navigation device carrying the underwater visual tactile sensor); the image acquisition unit 7 is hermetically fixed at the center inside the base 8 and is used to obtain the motion images of all the propeller units under the action of the fluid.

[0031] Specifically, the propeller units, the hemispherical shell 1, the middle cylindrical shell 2, and the base 8 are all formed by 9600 resin under the SLA stereolithography process, having good accuracy and stiffness, and can be placed in the underwater environment for a long time.

[0032] The passive propeller array designed in this embodiment bionics the fish lateral line structure, which can effectively visualize the fluid motion and is light and sensitive enough. Specifically, the passive propeller array has a total of 16 uniformly distributed propeller units. Among them, a propeller unit is arranged on the hemispherical shell 1 facing the image acquisition unit 7 (i.e., at the center of the hemispherical shell 1), and seven propeller units are uniformly arranged in a circle around the central propeller unit. The remaining eight propeller units are uniformly arranged on the spherical surface periphery of the hemispherical shell with the propeller unit located at the center of the hemispherical shell 1 as the center. The spacing between adjacent propeller units in the passive propeller array is uniform and does not interfere with each other.

[0033] Each of the aforementioned propeller units includes a propeller 9, a bearing 10 and a stepped shaft 11, and the propellers 9 are connected via the stepped shaft 11 to achieve synchronous rotation; one end of the stepped shaft 11 away from the propeller unit is limited within the hemispherical shell 1 and is within the field of view of the image acquisition unit 7.

[0034] The aforementioned stepped shaft 11 is provided with an identification label 111 on one end located in the hemispherical shell 1. When the identification label 111 rotates facing the propeller unit, the identification label 111 rotates synchronously, and the image acquisition unit 7 can capture the rotation image of each identification label 111; specifically, a circular chassis is provided on one end of the stepped shaft 11 located in the hemispherical shell 1, and an arrow pattern in the radial direction is provided on the circular chassis as an identification label 11. When the propeller 9 rotates and synchronously drives the stepped shaft to rotate, the arrow pattern also rotates synchronously, and the image acquisition unit can identify the label as a mark to identify the rotation angle of the propeller; more specifically, the stepped shaft is also formed by 9600 resin under the SLA stereo light curing process, and it has two steps, one of which is used to cooperate with the bearing 10, and the other step is used to cooperate with the propeller. The step design is to prevent interference between the bearing and the propeller.

[0035] The aforementioned bearing 10 uses a precision high-speed bearing with a very small radius, which can ensure the coaxiality of the propeller 9 and ensure the consistency of each rotation of the propeller 9. The small-sized bearing is mainly used to avoid the bearing size being too large to affect the water flow and facilitate installation. The specific size can be flexibly selected according to the actual application scenario.

[0036] An installation cavity 81 is provided in the aforementioned base 8, and an annular installation step 82 is provided on the periphery of the end of the installation cavity 81. A waterproof lens 4 is embedded and fixed on the installation step 82 to seal the image acquisition unit 7 in the base 8; and the connection between the middle cylindrical shell 2 and the base 8 is coated with a waterproof coating to form a sealed waterproof ring 6 to achieve double waterproofing.

[0037] In order to capture all the information of the propeller array as comprehensively as possible and prevent the exposure time from not matching the rotation speed of the propeller 9, a camera with a field of view (FOV) of 140 degrees is selected as the aforementioned image acquisition unit 7. The aforementioned camera consists of a Sony IMX258 lens module that supports autofocus and a motherboard that supports the UVC protocol, and the two are connected by a 3mm flat cable. It supports recording videos with a maximum resolution of 3840*2140 and a frame rate of 60fps, and can capture the slight rotation of the propeller 9 relatively quickly.

[0038] The aforementioned inertial measurement unit 3 (Inertial Measurement Unit, IMU) is used to make up for the problem that the propeller 9 is not sensitive enough to the tangential velocity. The IMU module uses a sensor of the ICM-42688-P model, which integrates a 3-axis gyroscope and a 3-axis accelerometer. The noise density of the gyroscope is as low as 4 mdps / √Hz; the noise density of the accelerometer is as low as 100 µg / √Hz.

[0039] In order to make up for the problem that the light is weak in the underwater environment, and the exposure time of the internal camera increases, resulting in the inability to clearly see the propeller array, in this embodiment, a ring-shaped light source 5 is provided at one end of the aforementioned middle cylindrical housing 2 close to the base 8, and the light of the ring-shaped light source 5 is projected onto the hemispherical housing 1; specifically, the ring-shaped light source 5 is a lamp ring composed of a series of 0603 patch LED lights. In order to ensure sufficient uniform illumination and no reflection, ten LED lights are evenly arranged in a ring, and after separate waterproof treatment, they are placed on the side of the waterproof lens 4 away from the camera.

[0040] The middle cylindrical housing 2 is also provided with a water outlet hole 21. A plurality of water inlet holes 12 arranged around the installation hole are provided outside the installation hole. After the external fluid acts on the propeller unit, it enters the underwater vision and touch sensor through the corresponding water inlet holes 12 and flows out from the water outlet hole 21. This can prevent the formation of turbulence of fluids such as water flow in the inner cavity of the underwater vision and touch sensor when it follows equipment such as an underwater vehicle. The continuous water outflow from the water outlet hole 21 can make the rotation of the propeller 9 more stable.

[0041] The aforementioned assembled whole underwater vision and touch sensor also needs to be waterproof designed. Specifically, a silicone waterproof gasket can be set in the installation gap between each housing and the connection unit, and then a layer of epoxy resin glue is filled outside the connection. After waiting for it to solidify, a black waterproof layer is formed. Then, it is poured with a transparent potting glue with an organic silicone component, and a dense transparent waterproof layer is formed after curing at room temperature. The silicone material can be disassembled under external force, which is convenient for maintenance while effectively protecting the internal electronic devices. The anti-fog design is to form a transparent coating layer on the surface of the waterproof lens 4 through an anti-fog agent, so that water vapor cannot condense on the surface, and desiccants are also arranged around the camera in the base 8 to ensure the long-term reliable use of the camera.

[0042] Embodiment 2 This embodiment provides an application method of the underwater vision and touch sensor provided in the aforementioned Embodiment 1. The steps of this method include: S1 The image acquisition unit 7 acquires the motion images of each propeller unit in the passive propeller array under the action of water flow; the inertial measurement unit 3 acquires the yaw angle of the underwater navigation device; S2 The motion images acquired in step S1 are enhanced to obtain enhanced image data; S3 Based on the enhanced image data, obtain the current angle value of the passive propeller array; perform differential processing on the angle values corresponding to the front and rear two frames of images to obtain the angular velocity of the passive propeller array; S4 Input the angular velocity into a convolutional neural network to predict the velocity estimation values of the underwater visual tactile sensor in the x-direction and y-direction; S5 Based on the velocity estimation values and the yaw angle, obtain the position of the underwater vehicle where the underwater visual tactile sensor is located.

[0043] The underwater visual tactile sensor based on fish lateral line bionics needs to effectively establish the correlation between water flow disturbance and body movement to achieve environment-independent underwater body movement estimation. The original data collected by the internal camera of the sensor is a multi-frame continuous image sequence, and the feature difference between consecutive multi-frames of images is very small. Directly using the pictures as the input of the neural network will cause a large amount of data redundancy. Moreover, a single image cannot represent the dynamic process of rotation, and only the frame difference method or the optical flow method can be used to process the image sequence, which will undoubtedly reduce its robustness in environments with light changes (such as direct sunlight) and water bubbles and floating impurities. For these reasons, in this embodiment, the original motion image sequence collected in step S1 is first preprocessed to extract the instantaneous angular velocity of each propeller 9 at the current moment, and then it is used as the input value of the neural network model to estimate the position of the vehicle body.

[0044] Specifically, in step S1, the image acquisition unit 7 acquires the rotation images of the identification tags 111 on each propeller unit as motion images. The underwater visual tactile sensor moves in the positive x and y directions in still water. According to the relationship of relative velocity, the absolute velocity of the underwater visual tactile sensor's movement is its velocity relative to the water flow. Therefore, the water flow will exert a certain force on the passive propeller array, causing the propeller 9 in the water flow direction to rotate under the action of the water flow force. The propellers 9 are distributed in all directions, and the water flow force in one direction generally drives the propellers 9 in multiple directions to rotate simultaneously. The rotational speed magnitudes of the propellers 9 in different directions can reflect the direction and magnitude of the water flow force. The action of the water flow is visualized as the rotation of a series of identification tags 111, and the rotation of the identification tags 111 is captured by the camera in the base.

[0045] The enhancement processing method in step S2 is: to ensure the consistency of the rotational speed estimation of multiple propellers 9, in OpenCV, the images of each propeller area in the motion image are cut out. In the propeller array, only the middle propeller is directly facing the camera, and the other propellers 9 form a certain angle with the camera. If the contours are directly recognized, the sizes of the identification tags 111 at different angles will be inconsistent, resulting in errors. Therefore, it is necessary to process the perspective change of the motion image through perspective transformation to transform the motion image into an angle directly facing the image acquisition unit. For the pixel points in the motion imageP , the pixel position conversion is performed using the following formula:

[0046] where P’ is the pixel position after transformation; H is the perspective matrix, which is obtained by solving the size parameters of the sensor housing model file.

[0047] In the aforementioned step S3, the method for obtaining the current angle value of the passive propeller array based on the enhanced image data is as follows: S301 identifies the label as an arrow pattern radially arranged on the circular chassis of the stepped shaft. The tip of the arrow serves as a pointer, pointing to the edge of the circular chassis; in order to remove the influence of impurities and image noise in the water and accurately obtain the rotation angle of the pointer , the Otsu algorithm is used to perform threshold segmentation on the image after the enhancement process in step S2 to obtain a binary image; S302 extracts the recognition label 111 from the binary image. Specifically, the tip of the arrow pattern is extracted from the binary image after threshold segmentation, and the rotation angle of the recognition label 111 is obtained , and is used as the angle value, then the angular velocity of the passive propeller array is expressed as:

[0048] where ω is the angular velocity, is the rotation angle in the k th frame of the image, is the rotation angle in the k+1 th frame of the image, f is the shooting frequency of the image acquisition unit (i.e., the camera).

[0049] In the aforementioned step S4, after obtaining the rotational speed of the propeller 9, a convolutional neural network is used to fit the complex non-linear mapping relationship between the rotational speed and the robot's movement speed. The convolutional neural network consists of two convolutional-pooling blocks, a fully connected layer, and an output layer. Each convolutional-pooling block contains a convolutional layer with a 1D convolutional kernel of 2, a ReLU activation function layer, and a pooling layer with a window of 2.

[0050] The input of the network is a vector composed of the angular velocities of 16 propellers 9. Due to various disturbances in the water flow, the rotation speed of a single propeller may be greatly affected, making it impossible to accurately reflect the overall movement speed of the robot. Therefore, the entire system adopts a redundant design. Considering the inherent relationship between the rotation speed of each unit on the propeller array and the overall spatial speed of the robot, even if the rotation speed of a certain propeller 9 is disturbed or abnormal, the rotation speed information of other propellers 9 can still effectively supplement and correct it, thus ensuring the accuracy of the overall speed estimation. Convolution and pooling operations are used to aggregate the rotation speed information of different propellers 9. After two convolution-pooling blocks and a flattening layer, the input is converted into a 128-dimensional feature D containing all the propeller rotation speed information. Feature D enters two Dense layers with 64 units and 2 units respectively, and finally outputs a two-dimensional vector V, which is the linear velocity vector along the forward direction of the sensor and the linear velocity vector perpendicular to the forward direction on the horizontal plane.

[0051] In the aforementioned step S5, since the passive propeller array is sensitive to horizontal movement, but the change of the propeller 9 is not significant when the sensor body rotates. And the hydrodynamic model under the condition of the body rotation is more complex, and the movement of the propeller 9 is difficult to predict. Therefore, a gyroscope is installed on the middle cylindrical housing 2. By integrating the attitude quaternion of the vehicle body through the gyroscope, the Yaw angle is obtained. According to the estimated Yaw angle and the previously estimated velocities in the x and y directions, a complete estimation of the body position on the plane can be made.

[0052] Specifically, the passive propeller array is sensitive to the horizontal movement of the underwater visual and tactile sensor, but the change of each propeller is not significant when the underwater visual and tactile sensor body rotates. And the hydrodynamic model under the condition of the underwater visual and tactile sensor body rotation is more complex, and the movement of the propeller is difficult to predict. Therefore, this underwater visual and tactile sensor can only estimate the linear velocities in the x and y directions on the plane and cannot be used to estimate the angular velocity. So this application needs to set an IMU or a gyroscope as an inertial measurement unit to obtain the angular velocity, and then achieve a complete estimation of the accurate positions of the underwater visual and tactile sensor body and the underwater vehicle body on the plane.

[0053] More specifically, assume that the z-axis angular velocity obtained through the IMU module at the current moment is , that is, the Yaw angle (i.e., the orientation angle at the current moment) can be obtained through integration. The specific formula is:

[0054] yaw( t ) is t the orientation angle at time t-1) is t the orientation angle at time - 1, ∆t is the sampling time interval. Assuming the motion of the object on the plane satisfies rigid - body kinematics, its position update equation is as follows:

[0055] where 、 are the velocities at the current time, respectively, is the orientation angle at the current time, are respectively the linear velocities in the x, y direction at the current time, that is, the velocities estimated by the visual - tactile sensor. The linear velocities here are in the object coordinate system rather than the global coordinate system. The angle update equation is:

[0056] where, is the angle at the current time, is the angular velocity of the Yaw angle, obtained through the IMU or gyroscope.

[0057] It should be understood that expressions such as "including" and "may include" that can be used in this application indicate the existence of the disclosed functions, operations, or constituent elements, and do not limit one or more additional functions, operations, and constituent elements. In this application, terms such as "including" and / or "having" can be interpreted as indicating a specific characteristic, number, operation, constituent element, component, or a combination thereof, but cannot be interpreted as excluding the existence or possibility of addition of one or more other characteristics, numbers, operations, constituent elements, components, or a combination thereof.

[0058] In addition, in this application, the expression "and / or" includes any and all combinations of the associated listed words. For example, the expression "A and / or B" can include A, can include B, or can include both A and B.

[0059] In the description of the embodiments of the present application, it should be noted that unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense. For example, "connection" can be a detachable connection or a non-detachable connection; it can be a direct connection or an indirect connection through an intermediate medium. Among them, "fixed connection" means that the two are connected and the relative position relationship after connection remains unchanged. "Rotational connection" means that the two are connected and can rotate relative to each other after connection. "Sliding connection" means that the two are connected and can slide relative to each other after connection. The orientation terms mentioned in the embodiments of the present application, such as "top", "bottom", "inner", "outer", "left", "right", etc., are only references to the direction of the accompanying drawings. Therefore, the orientation terms are used to better and more clearly illustrate and understand the embodiments of the present application, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the embodiments of the present application.

[0060] In addition, in the embodiments of the present application, mathematical concepts such as symmetry, equality, parallelism, and perpendicularity are mentioned. These limitations are all with respect to the current technological level, rather than an absolutely strict definition in the mathematical sense. A small deviation is allowed, and being approximately symmetric, approximately equal, approximately parallel, approximately perpendicular, etc. are all acceptable. For example, when it is said that A is parallel to B, it means that A and B are parallel or approximately parallel, and the included angle between A and B can be between 0 degrees and 10 degrees. When it is said that A is perpendicular to B, it means that A and B are perpendicular or approximately perpendicular, and the included angle between A and B can be between 80 degrees and 100 degrees.

[0061] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An underwater visual and tactile sensor based on the bionic of fish lateral line, characterized in that It includes a hollow housing, an inertial measurement unit (3), an image acquisition unit (7) and a propeller unit; the hollow housing includes a hemispherical housing (1), a middle cylindrical housing (2) and a base (8) connected in sequence. A plurality of mounting holes arranged in an array are formed in the hemispherical housing (1), and the propeller units are fixed in the corresponding mounting holes and form a passive propeller array protruding from the hemispherical housing (1); the inertial measurement unit (3) is arranged outside the middle cylindrical housing (2) and is used to measure the yaw angle of the underwater visual tactile sensor; the image acquisition unit (7) is hermetically fixed at the center inside the base (8) and is used to acquire the motion images of all propeller units under the action of fluid.

2. The underwater visual and tactile sensor according to claim 1, wherein The propeller unit includes a propeller (9), a bearing (10) and a stepped shaft (11). The propeller (9) is connected to the bearing (10) through the stepped shaft (11), and the propeller (9) and the stepped shaft (11) can rotate synchronously; one end of the stepped shaft (11) away from the propeller (9) is limited inside the hemispherical housing (1) and is within the field of view angle of the image acquisition unit (7).

3. The underwater visual and tactile sensor according to claim 2, wherein An identification tag (111) is provided at one end of the stepped shaft (11) inside the hemispherical housing (1), and the identification tag (111) can rotate synchronously with the propeller (9).

4. The underwater visual and tactile sensor according to claim 2, wherein The propeller (9) is a three-blade propeller; and / or, one of the three-blade propellers is facing the image acquisition unit (7).

5. The underwater visual tactile sensor according to claim 1, characterized in that, An installation cavity (81) is provided inside the base (8), and an installation step (82) is provided at the peripheral edge of the end of the installation cavity (81). A waterproof lens (4) is fixed on the installation step (82) to seal the image acquisition unit (7) inside the base (8); and / or, a waterproof coating (6) is applied at the connection between the middle cylindrical housing (2) and the base (8).

6. The underwater visual and tactile sensor according to claim 1, wherein A ring light source (5) is provided at one end of the middle cylindrical housing (2) close to the base (8), and the light of the ring light source (5) irradiates the hemispherical housing (1); and / or, a water outlet hole (21) is provided on the middle cylindrical housing (2), and a water inlet hole (12) is provided outside the mounting hole. After the external fluid acts on the propeller unit, it enters the underwater visual tactile sensor through the corresponding water inlet hole (12) and flows out from the water outlet hole (21).

7. An application of the underwater visual tactile sensor according to any one of claims 1-6, characterized in that, It includes: S1 The image acquisition unit (7) acquires the motion images of each propeller unit in the passive propeller array under the action of water flow; the inertial measurement unit (3) acquires the yaw angle of the underwater visual tactile sensor; S2 Perform enhancement processing on the motion images acquired in step S1 to obtain enhanced image data; S3 Based on the enhanced image data, obtain the current angle value of the passive propeller array; Perform differential processing on the angle values corresponding to the front and rear two frames of images to obtain the angular velocity of the passive propeller array; S4 Input the angular velocity into a convolutional neural network to obtain the speed estimation values of the underwater visual tactile sensor in the x direction and the y direction; S5 obtains the position of the underwater visual tactile sensor based on the speed estimation value and the yaw angle.

8. The application according to claim 7, wherein In step S1, the image acquisition unit (7) acquires the rotation image of the identification tag (111) on each propeller (9) as the motion image.

9. The application according to claim 7, characterized in that In step S2, the enhancement process is to process the perspective change of the moving image through perspective transformation, convert the moving image into an angle directly facing the image acquisition unit, and for the pixel points in the moving image P , the pixel position conversion is performed using the following formula: Among them, P’ is the transformed pixel position; H is the perspective matrix.

10. The application according to claim 7, characterized in that In step S3, the method for obtaining the current angle value of the passive propeller array based on the enhanced image data is as follows: S301 performs threshold segmentation on the enhanced image data to obtain a binary image; S302 extracts the recognition label (111) from the binary image and obtains the rotation angle of the recognition label (111). As the angle value, the angular velocity of the passive propeller array is expressed as: where ω is the angular velocity, is the rotation angle in the k th frame of the image, is the rotation angle in the k+1 th frame of the image, f and is the shooting frequency of the image acquisition unit.

Images