Method and system for sensing flow velocity and flow direction of autonomous vision bionic lateral line sensor

Abstract

The invention discloses an autonomous vision bionic lateral line sensor flow velocity and flow direction sensing method and system. The method comprises the following steps: building a bionic lateral line sensing deployment platform which comprises a visual bionic lateral line sensor and is used for sensing and measuring the direction and the speed of a flow field; the neural network modeling part is used for establishing a flow field sensing model of the bionic lateral line sensor, the neural network modeling part is used for processing complex data in a flow field and outputting estimated values of flow velocity and flow direction on the basis of sensing and measuring results of the visual bionic lateral line sensor, and the whisker deformation physical modeling part is used for modeling the visual bionic lateral line sensor. Deducing the correlation between the whisker deformation and the flow velocity and flow direction through the function relationship; and performing weighted combination on an estimated value output by the neural network modeling part and a derivation result output by the whisker deformation modeling part to obtain a flow velocity and flow direction comprehensive result. According to the invention, the sensing capability of multi-directional flow velocity and flow direction in a complex flow field can be effectively improved, more environmental requirements can be met, and the precision and robustness of the system can be improved.

Description

Technical Field

[0001] The invention belongs to the technical field of underwater sensors, and in particular relates to a flow velocity and direction sensing method of an autonomous visual bionic lateral line sensor. Background Art

[0002] The bionic lateral line sensor is one of the key components for underwater robots to sense flow fields, and has broad application prospects in underwater robot perception and control. After the bionic lateral line sensor comes into contact with the fluid, it can convert the water flow information into electrical signals or image signals according to different perception principles. Then, the data collected by the bionic lateral line sensor is processed, and the water flow information is extracted through modeling calibration or data fitting. Compared with one-dimensional electrical signals, two-dimensional image signals contain more information. By capturing the deformation of the sensing tentacles through the visual system, the single-point two-dimensional omnidirectional water flow information can be retained, and more efficient and stable water flow sensing can be performed.

[0003] When analyzing the deformation of the contact surface, the traditional visual processing algorithm mainly estimates the external force by calculating the position offset of the feature points. It has high requirements for recognition accuracy and low generalization, which makes it unfavorable for complex and harsh underwater working environments. With the development of deep learning technology, deep neural networks are used to process the image information of the contact surface to realize the deformation perception of the elastic body, without the need to design complex visual algorithms and with better generalization performance. However, there are still the following problems: it can only collect flow velocity data in a fixed direction and cannot detect the flow velocity in a non-specified direction; it does not have the ability to perceive the flow direction; it requires high experience in debugging parameters and has a limited adjustment range; the learning accuracy of a single neural network is insufficient, resulting in insufficient perception accuracy. Therefore, how to apply neural networks to bionic lateral line sensors to process two-dimensional image information to improve the stability and generalization of water flow perception has become a problem that needs to be solved urgently. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an autonomous visual bionic lateral line sensor flow velocity and direction perception method and system, which can effectively improve the perception ability of multi-directional flow velocity and direction in complex flow fields, adapt to more environmental requirements and improve the accuracy and robustness of the system.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] A flow velocity and direction sensing method using an autonomous visual bionic lateral line sensor, comprising:

[0007] Building a bionic lateral line sensing deployment platform, including a host computer and a visual bionic lateral line sensor, wherein the visual bionic lateral line sensor is used to sense and measure the direction and speed of the flow field;

[0008] Establishing a flow field perception model of a bionic lateral line sensor, including a neural network modeling part and a tentacle deformation physical modeling part, wherein the neural network modeling part processes complex data in the flow field and outputs an estimated value of the flow velocity and direction based on the perception and measurement results of the visual bionic lateral line sensor, and the tentacle deformation physical modeling part models the visual bionic lateral line sensor and derives the correlation between the tentacle deformation and the flow velocity and direction through a functional relationship;

[0009] The estimated value output by the neural network modeling part and the derived result output by the tentacle deformation modeling part are weightedly combined to obtain the comprehensive result of flow velocity and direction.

[0010] On the other hand, the present invention provides an autonomous visual bionic lateral line sensor flow velocity and direction perception system, comprising:

[0011] A construction unit, used for constructing a bionic lateral line sensing and deployment platform, including a host computer and a visual bionic lateral line sensor, wherein the visual bionic lateral line sensor is used for sensing and measuring the direction and speed of the flow field;

[0012] A model building unit, for establishing a flow field perception model, including a neural network modeling part and a tentacle deformation physical modeling part, wherein the neural network modeling part processes complex data in the flow field and outputs an estimated value of the flow velocity and direction based on the perception and measurement results of the visual bionic lateral line sensor, and the tentacle deformation physical modeling part models the visual bionic lateral line sensor and derives the correlation between the tentacle deformation and the flow velocity and direction through a functional relationship;

[0013] The result output unit is used to perform weighted combination of the estimated value output by the neural network modeling part and the derived result output by the tentacle deformation modeling part to obtain a comprehensive result of flow velocity and direction.

[0014] In the third part, the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned autonomous visual bionic lateral line sensor flow velocity and direction perception method.

[0015] In the fourth part, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned autonomous visual bionic lateral line sensor flow velocity and direction perception method.

[0016] The beneficial effects of the present invention are:

[0017] The present invention realizes the perception of flow velocity and direction through the offset of flexible tentacles, combines image feature extraction network, time series feature extraction network and tentacle deformation physical modeling to perform synchronous estimation of flow velocity and flow direction in the flow field, can effectively realize high-precision detection of flow velocity and direction, and has good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a bionic lateral line sensing deployment platform of the present invention;

[0019] Figure 2 It is a schematic diagram of the structure of a single-whisper bionic lateral line sensor device;

[0020] Figure 3 This is a schematic diagram of a flow velocity and direction sensing method of an autonomous visual bionic lateral line sensor of the present invention;

[0021] Figure 4 This is a flow chart of a flow velocity and direction sensing method of an autonomous visual bionic lateral line sensor of the present invention;

[0022] Figure 5 Training flowchart for the neural network modeling part;

[0023] Figure 6 The network structure diagram for the physical modeling of tentacle deformation;

[0024] Figure 7 This is a schematic diagram of the data set structure;

[0025] Figure 8 Flowchart for single-whisker dataset acquisition. DETAILED DESCRIPTION

[0026] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0027] like Figure 1As shown, the experimental environment in which the flow velocity and direction perception method of an autonomous visual bionic lateral line sensor of the present invention is executed, a bionic lateral line perception deployment platform, including the functional modules in the host computer and the visual bionic lateral line sensor, controllable flow velocity propeller, sensor rotating device, rectifier tube, water storage container, and water flow velocity control device required for the actual implementation plan, and the visual bionic lateral line sensor is located in the test area. The water storage container is used to construct the basic experimental environment; the rectifier tube and the upper plate of the test area are used to construct the experimental test area so as to form a relatively stable water flow in a rectangular area in the water tank; the controllable flow velocity propeller is placed on one side of the rectifier tube, and the water flow velocity control device is placed on the wall and controllably connected to the controllable flow velocity propeller to provide a stable water velocity for the rectifier tube; one end of the sensor rotating device is fixed to the upper end of the visual lateral line sensor to rotate the sensor to indicate the direction of the water flow, and the other end is connected to the host computer through a data line to control the specific rotation angle.

[0028] The visual side line sensor, such as Figure 2 As shown: It includes a single-whisper bionic lateral line sensor (the diagram is simple, expressed as a single-whisper sensor), which is the basis for measuring the direction and speed of the flow field. It must be able to accurately measure the two quantities of flow velocity and flow direction in the constant flow field, including flexible tentacles and a hard sleeve shell of the tentacle. Based on the above test requirements, the single-whisper experimental device needs to be able to meet the two functions of position fixation and angle rotation of the sensor. The tentacle of the single-whisper bionic lateral line sensor is made of soft transparent silicone, the root of the tentacle is round, and the outside of the tentacle is a hard sleeve shell of the same shape with a round root.

[0029] In one example, the serial port servo is fixed to the single-tentacled bionic lateral line sensor bracket by screws, and the servo turntable is then screwed to the upper part of the sensor, so that the sensor can be fixed at the appropriate depth and correct position. The opening of the upper cover of the test area is on the same axis as the servo turntable, and the diameter of the upper cover opening is the same as the diameter of the sensor, which ensures that the test area is relatively sealed without water overflow while limiting the sensor. The servo can achieve 0~360° rotation with a rotation accuracy of less than 0.1°, which meets the requirements of the sensor for image acquisition and testing of flows in different directions. The signal and power supply of the servo are provided by the CH340 to RS485 signal adapter board and an independent 12V DC power supply, where the signal adapter board is directly connected to the test host computer to complete the entire device.

[0030] The host computer connects the single-whisker bionic lateral line sensor and the sensor rotating device to realize image transmission and rotation command transmission of the sensor rotating device.

[0031] The water flow rate control device is used to drive the controllable flow rate propeller to control the flow rate value of the water flow. The water flow rate is linearly related to the motor speed of the controllable flow rate propeller, so the motor speed can be used to represent the water flow rate. In one example, the water flow rate control device is driven by an AC 220V adjustable speed motor, and its speed can be adjusted between 0 and 1000 rpm. The rotation of the controllable flow rate propeller controlled by it can generate a water flow at a certain speed. The water flow passes through the rectifying and stabilizing channel to achieve the purpose of eliminating eddy currents, stabilizing speed, and reducing bubbles, and then enters the detection area to be captured and measured by the sensor.

[0032] The rectifier tube is used to provide a stable water flow.

[0033] The sensor rotating device is used to simulate the direction of water flow.

[0034] The upper plate of the test area is designed to be detachable and is compressed and sealed with the remaining parts by bolts, so that the test area forms a sealed small water tunnel experimental device.

[0035] Based on the above settings, the present invention provides a flow velocity and direction perception method of an autonomous visual bionic lateral line sensor, which focuses on combining a neural network model with a tentacle deformation model to establish a flow field perception model of a bionic lateral line sensor, thereby achieving accurate perception and analysis of flow velocity and direction. The implementation process of the present invention is described in detail below in conjunction with specific steps.

[0036] like Figure 3 As shown, the flow field perception model includes a neural network modeling part and a tentacle deformation physical modeling part. The two parts are combined to achieve comprehensive perception of the flow field. By weighted combination of the output results of the neural network modeling part and the results of the tentacle deformation modeling part, a comprehensive result is finally obtained for accurate perception and analysis of the flow field. The neural network model in the method is obtained by training based on the neural network learning method, and is used to process complex data in the flow field and output estimation results of flow velocity and direction. At the same time, the tentacle deformation physical model is obtained by constructing a functional relationship based on physical quantities obtained by image detection. The relationship between tentacle deformation and flow velocity and direction is derived through the functional relationship, thereby achieving more accurate flow field perception and analysis.

[0037] Specifically, the neural network modeling part includes an image feature extraction neural network and a time series feature extraction neural network: the image feature extraction neural network includes a convolutional neural network, which is used to extract image feature data in complex flow fields; the time series feature extraction neural network includes a long short-term memory network, which is used to process the time series feature data of images. Through the coordinated action of the two, the estimated results of flow velocity and flow direction are output.

[0038] The tentacle deformation modeling part includes a flow field tentacle deformation model, which is obtained by modeling the deformation of the tentacle based on the physical quantities such as the water velocity and direction obtained by the detection image deformation and the controllable flow velocity instrument. The flow field tentacle deformation model can reflect the functional relationship between the tentacle deformation and the water velocity and direction. This part directly reflects the changes in the local flow field through the deformation of the local tentacles.

[0039] It should be noted that, optionally, in some special cases, the neural network part may also use other image feature extraction networks and time series feature extraction networks to adapt to more complex flow field environments. For example: the residual network, as an image feature extraction network, is suitable for deep network structures and can better capture tiny features in the flow field. The multi-layer residual structure of the residual network helps to extract more representative features in complex flow fields, and is suitable for flow field analysis with complex turbulence or multi-directional flow. The gated recurrent unit, as a time series feature extraction network, can replace the long short-term memory network in some cases. Compared with the long short-term memory network, the gated recurrent unit has a simpler structure and higher computational efficiency. It performs better when processing simple time series data and is particularly suitable for rapid response to real-time flow field changes.

[0040] According to the present invention, a flow velocity and direction perception method of an autonomous visual bionic lateral line sensor is provided, which can be applied to a perception model training system and platform.

[0041] Specifically, Figure 4 As shown, first input the color image, then calculate the coordinates of the area to be detected, crop the input color image into a square area slightly larger than the feature area, and input the cropped image into the trained neural network modeling part to obtain result 1, which is recorded as , the data in the same test set is input into the established flow field tentacle deformation model to obtain the second result, which is recorded as , the weighted proportional coefficient is ɑ, and its value is between [0,1]. Then the final flow velocity and direction estimation result is It can be expressed by the following formula:

[0042] ,

[0043] Among them, ɑ controls the weights of the neural network modeling part and the tentacle deformation modeling part. When ɑ=1, the result depends entirely on the output of the neural network modeling part; when ɑ=0, the result depends entirely on the output of the tentacle deformation modeling part. The final flow velocity and direction estimation results are obtained. , which contains the sine and cosine values ​​representing the direction of the water flow and the flow rate of the water flow.

[0044] It should be noted that other weights ɑ can be selected. For example, when the effect of the tentacle deformation modeling part is poor, the weight ɑ can be appropriately increased. When the result loss of the neural network modeling part is too large, the weight ɑ can be appropriately reduced.

[0045] Among them, the process of inputting the image into the flow field tentacle deformation model is as follows: the cropped image is separated from the red channel, and then its red contrast is stretched. The threshold method is used to find the position offset of the center of the black area at the root of the tentacle shell relative to the center of the tentacle. According to the functional relationship obtained by modeling the flow field tentacle deformation model, the flow velocity and direction from the tentacle deformation model are output.

[0046] like Figure 5 As shown, it is a training flow chart of the neural network modeling part of the present invention. First, a training sample is obtained through the image information collected by the bionic lateral line sensor, and the training sample includes a first input value;

[0047] The image is preprocessed on the host computer, and a small part of the collected image is preprocessed to obtain the average value of the grayscale extreme value and the average value of the coordinates of the grayscale circle at the base of the tentacle. The identification of the grayscale circle at the base of the tentacle adopts the Hough circle recognition technology, and finally the average value of the positioning information of the two-dimensional center of the tentacle root and the average value of the grayscale extreme value are obtained. Since the image taken by the camera may have invalid features, the image is cropped into an image slightly larger than the effective feature area according to the center position of the circle.

[0048] The image preprocessing is to input a first input value and obtain first, second and third output values. The first output value refers to the cropped image. In one example, the small image is cropped to 100. 100 pixels image.

[0049] The second output value is the grayscale extreme value of the cropped image, and the third output value refers to the coordinates of the center of the grayscale circle at the base of the tentacle.

[0050] Based on the first, second, and third output values ​​and the training samples, the neural network modeling part is trained to determine the loss function between the network result output value and the true value of the label.

[0051] The label value of the neural network training is determined by the sensor rotating device and the water flow rate control device (see Figure 2 ) is provided, the sensor rotation device is used to simulate the direction of water flow, and the water flow direction value is provided by using a functional relationship; the water flow velocity control device is used to control the water flow velocity value; the water flow direction value and the water flow velocity value are jointly processed into the true value of the label of the training sample.

[0052] In one example, in the network training part, a combined model of a convolutional neural network and a long short-term memory network is used for training, and the mean square error is used as the loss function of the training to obtain a trained network model.

[0053] Therefore, the training samples include input values ​​and true label values, the input values ​​are a series of image frames, and the training set data includes sample values ​​required for autonomous visual bionic lateral line sensor training at different angles under different flow rates.

[0054] The average value of the grayscale extreme value of the image is derived according to the second and third output values. The network model is trained based on the two auxiliary parameters and the training set data. The subsequent images are subjected to regional coordinate calculation according to the third output value, i.e., the average value of the center coordinate of the grayscale circle, and the images are cropped to images slightly larger than the size of the feature area, and then the neural network is trained; finally, the trained neural network model is output.

[0055] Specifically, the network model can be a combination of a convolutional neural network and a long short-term memory network, wherein the convolutional neural network and the long short-term memory network are composed of two convolutional layers, two pooling layers, two fully connected layers and a long short-term memory network layer.

[0056] The input of the network structure is a sequence of several photos. Before the photos are pushed into the network, they are grayed out and then pushed into the sequence. Each time before pushing into the sequence, the first photo is ejected and a new photo is pushed into the network. The output of the network is the flow rate and direction of the water.

[0057] Under the similar network structure, compared with the convolutional neural network combined with the long short-term memory network, a single convolutional neural network may not be able to accurately capture the characteristics of the image when the change in the tentacle deformation amplitude is small, and therefore cannot effectively extract and perceive the characteristic information of the flow velocity and flow direction in the fluid-solid interface. The model obtained by combining the convolutional neural network with the long short-term memory network is superior to the model obtained by a single convolutional neural network in both prediction accuracy and sensitivity due to the introduction of time series information. It has a more accurate perception of flow velocity, a more stable prediction result for flow direction, and can better track a larger angle range. It can distinguish the flow velocity and flow direction through vibration information when the change in sensor deformation is not obvious, thus proving the role of time series information in improving accuracy and sensitivity.

[0058] like Figure 6 As shown, this is the flow chart for establishing the physical modeling part of tentacle deformation.

[0059] The first output value, the second output value, and the third output value are input into the physical modeling part of the tentacle deformation based on the bionic lateral line sensor to find the functional relationship between the displacement offset and the angle between the center of the circular hard sleeve shell at the base of the tentacle and the center of the base of the tentacle. The first output value refers to the cropped image, the second output value refers to the grayscale extreme value of the cropped image, and the third output value refers to the coordinates of the center of the grayscale circle at the base of the tentacle.

[0060] The label value of the physical modeling part of the tentacle deformation, that is, the flow velocity and flow direction of the water flow, is obtained by the bionic lateral line sensor flow velocity and flow direction sensing device.

[0061] The tentacle deformation model is in the upper computer, which calculates the cropped image, positioning information, and grayscale extreme value from the preprocessing module, and then finds the small circle of the hard cover shell at the root of the tentacle (see Figure 2 ) and calculate the difference between the coordinates of the center of the tentacle root and the center of the hard cover shell at the root of the tentacle. , find the functional relationship between the flow velocity and direction and the deformation of the tentacle in the picture. In the process of finding the center of the circular hard cover shell at the base of the tentacle, the method of calculating the grayscale extreme red channel separation and then stretching the red contrast is adopted. The threshold method is then used to identify the coordinates of the center of the bottom of the hard cover shell at the base of the tentacle, and the coordinates of the center of the circle are obtained, and a set of average values ​​are calculated.

[0062] In one example, an image acquisition processing method is as follows: Figure 8 As shown, a single whisker sensor can collect data in real time. The data set structure is as follows: Figure 7 First, the sensor is initialized, and the center of the circular spot area is determined by identifying the circular spot formed by the dark position of the tentacles in the image, and a color image slightly larger than the detection area is cropped with this point as the center of the image.

[0063] In one example, 100 100px color image.

[0064] After receiving the storage instruction, the program automatically reads the target angle and controls the digital servo to rotate to the target position according to the pre-entered angle sequence. Then, the program reads the speed information in the sequence and initializes the image register and the save identifier.

[0065] After completing the above steps, enter the image capture section.

[0066] In one example, considering the hardware performance of the sensor, computer processing power and other factors, the image data sampling rate of the sensor is set to 20 times per second, and the number of images in each continuous sequence is 500. After each round of acquisition, the program sends the next position to the servo through the serial port and delays for 8 seconds so that the image collected by the sensor is stable. The entire process does not require manual intervention except adjusting the motor speed to change the flow rate, and can automatically perform different flow direction image information acquisition tasks.

[0067] After the image is collected, it is first stored in a temporary array. After all the images are collected, they are stored in the root directory of the dataset in the order of flow speed and flow direction in png format. The dataset structure is as follows: Figure 7 shown.

[0068] The flow rate is linearly related to the motor speed, so the water flow rate can be expressed by the motor speed, and the flow direction corresponds to the corresponding angle of the steering gear control, ranging from 0 to 4096. After the acquisition is completed, the program automatically creates labels for the pictures, and then stores them uniformly in npy format files with "label+speed+angle.npy" as the format template. Compared with storing the label of each photo separately, it reduces the number of small files, optimizes the data structure of the data set, and improves the reading speed of the data set content in actual use.

[0069] In one example, according to the experimental device conditions, the data set covers 7 speeds ranging from 250rpm to 550rpm with an accuracy of 50rpm. At each speed, image data at 30 different angles within a 360° range are collected, with a total of 105,000 image files.

[0070] In one example, the division and processing of the single-tentacle image dataset follows the general network training dataset division method, dividing the training set and the test set in a ratio of 9:1. The specific method is to first number the entire set starting from 0, then traverse the file to obtain the number of small datasets at different angles and speeds that the entire set is divided into, and finally select the corresponding dataset as the test set by randomly generating numbers, and the remaining part as the training set. After the division, the dataset is renumbered and placed in the corresponding training folder.

[0071] The long sequence images themselves are continuous in the changes of flow speed and flow direction, so they can test the effect of the model on the regression task. Therefore, for the validation set of this training, several longer continuous sequences are selected as the validation set of images.

[0072] On the other hand, the present invention provides an autonomous visual bionic lateral line sensor flow velocity and direction perception system, the various units included in the system can implement the aforementioned various steps, including:

[0073] A construction unit, used for constructing a bionic lateral line sensing and deployment platform, including a host computer and a visual bionic lateral line sensor, wherein the visual bionic lateral line sensor is used for sensing and measuring the direction and speed of the flow field;

[0074] A model building unit, for establishing a flow field perception model, including a neural network modeling part and a tentacle deformation physical modeling part, wherein the neural network modeling part processes complex data in the flow field and outputs an estimated value of the flow velocity and direction based on the perception and measurement results of the visual bionic lateral line sensor, and the tentacle deformation physical modeling part models the visual bionic lateral line sensor and derives the correlation between the tentacle deformation and the flow velocity and direction through a functional relationship;

[0075] The result output unit is used to perform weighted combination of the estimated value output by the neural network modeling part and the derived result output by the tentacle deformation modeling part to obtain a comprehensive result of flow velocity and direction.

[0076] In the third part, the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned autonomous visual bionic lateral line sensor flow velocity and direction perception method.

[0077] In the fourth part, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned autonomous visual bionic lateral line sensor flow velocity and direction perception method.

[0078] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A flow velocity and direction sensing method using an autonomous visual bionic lateral line sensor, characterized in that: include: Building a bionic lateral line sensing deployment platform, including a host computer and a visual bionic lateral line sensor, wherein the visual bionic lateral line sensor is used to sense and measure the direction and speed of the flow field; Establishing a flow field perception model, including a neural network modeling part and a tentacle deformation physical modeling part, wherein the neural network modeling part processes complex data in the flow field and outputs an estimated value of the flow velocity and direction based on the perception and measurement results of the visual bionic lateral line sensor, and the tentacle deformation physical modeling part models the visual bionic lateral line sensor and derives the relationship between the tentacle deformation and the flow velocity and direction through a functional relationship; The estimated value output by the neural network modeling part and the derived result output by the tentacle deformation modeling part are weightedly combined to obtain the comprehensive result of flow velocity and direction.

2. The method for sensing flow velocity and direction using an autonomous visual bionic lateral line sensor according to claim 1 is characterized in that: The bionic lateral line sensing deployment platform also includes a sensor rotating device and a water flow rate control device. One end of the sensor rotating device is fixed to the visual bionic lateral line sensor, and the other end is connected to the host computer. The sensor rotating device is rotated according to the rotation angle in the host computer instruction to indicate the direction of the water flow; the water flow rate control device is connected to the visual bionic lateral line sensor to control the speed of the water flow.

3. The method for sensing flow velocity and direction using an autonomous visual bionic lateral line sensor according to claim 2 is characterized in that: The visual bionic lateral line sensor is a single-tentacle bionic lateral line sensor, including a flexible tentacle and a tentacle hard cover shell. The tentacle is made of soft transparent silicone, and the root of the tentacle is round. The tentacle hard cover shell is a round hard cover shell with the same shape as the root of the tentacle.

4. The method for sensing flow velocity and direction using an autonomous visual bionic lateral line sensor according to claim 1, characterized in that: The neural network modeling part includes an image feature extraction neural network and a time series feature extraction neural network. The image feature extraction neural network is used to extract image feature data in a complex flow field, and the time series feature extraction neural network is used to process the time series feature data of an image.

5. The method for sensing flow velocity and direction using an autonomous visual bionic lateral line sensor according to claim 3 is characterized in that: The image feature extraction neural network includes any one of a convolutional neural network and a residual network; the time series feature extraction neural network includes any one of a long short-term memory network and a gated recurrent unit.

6. The method for sensing flow velocity and direction using an autonomous visual bionic lateral line sensor according to claim 1, characterized in that: The tentacle deformation physical modeling part includes a flow field tentacle deformation model. The flow field tentacle deformation model obtains the water flow velocity and flow direction based on the detection image deformation and the water flow velocity control device, and models the tentacle deformation of the visual bionic lateral line sensor.

7. The method for sensing flow velocity and direction using an autonomous visual bionic lateral line sensor according to claim 4, characterized in that: The neural network modeling part specifically includes: Collecting image information through the bionic lateral line sensor to obtain training samples; Preprocessing the training sample to obtain first, second and third output values, wherein the first output value is a cropped image of the training sample, the second output value is a grayscale extreme value of the cropped image, and the third output value is a coordinate value of a center of a grayscale circle at the base of the tentacle; Based on the first, second, and third output values ​​and the training samples, the neural network modeling part is trained to determine the loss function between the network result output value and the true value of the label to obtain a trained network model.

8. The method for sensing flow velocity and direction using an autonomous visual bionic lateral line sensor according to claim 4 is characterized in that: The tentacle deformation physical modeling part specifically includes: The first, second and third output values ​​are input into the tentacle deformation physical modeling part to find the functional relationship between the displacement offset and the angle between the center of the circular hard sleeve shell at the base of the tentacle and the center of the tentacle root; wherein finding the center of the circular hard sleeve shell at the base of the tentacle includes first calculating the grayscale extreme value, then separating the red channel of the image, and then stretching the red contrast, and using the threshold method to identify the coordinates of the center of the hard sleeve shell at the base of the tentacle.

9. An autonomous visual bionic lateral line sensor flow velocity and direction perception system, characterized in that: include: A construction unit, used for constructing a bionic lateral line sensing and deployment platform, including a host computer and a visual bionic lateral line sensor, wherein the visual bionic lateral line sensor is used for sensing and measuring the direction and speed of the flow field; A model building unit, for establishing a flow field perception model, including a neural network modeling part and a tentacle deformation physical modeling part, wherein the neural network modeling part processes complex data in the flow field and outputs an estimated value of the flow velocity and direction based on the perception and measurement results of the visual bionic lateral line sensor, and the tentacle deformation physical modeling part models the visual bionic lateral line sensor and derives the correlation between the tentacle deformation and the flow velocity and direction through a functional relationship; The result output unit is used to perform weighted combination of the estimated value output by the neural network modeling part and the derived result output by the tentacle deformation modeling part to obtain a comprehensive result of flow velocity and direction.

10. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; Wherein, when one or more programs are executed by the one or more processors, the one or more processors implement the flow velocity and direction perception method of an autonomous visual bionic lateral line sensor as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by the processor, the processor can implement the flow velocity and direction perception method of an autonomous visual bionic lateral line sensor as described in any one of claims 1-8.

Images