Three-dimensional model reconstruction method, device and equipment for dynamic fishes in fish passing channel and medium

Abstract

The invention discloses a three-dimensional model reconstruction method and device for dynamic fishes in a fish passing channel, equipment and a medium, and relates to the technical field of river ecological monitoring. According to the method, restoration of 1: 1 real fish data is realized based on a depth image technology, and swimming fish depth image data is converted into linear state data (namely, dynamic fish is converted into linear static fish) based on a deformation correction method, so that three-dimensional reconstruction is completed. Complete and accurate geometric parameters of the fish body can be obtained based on the three-dimensional reconstruction model of the fish, and the features of the fish body can be further analyzed and extracted. According to the invention, the accuracy of body size measurement and feature extraction is improved.

Description

Technical Field

[0001] The present application relates to the technical field of river ecological monitoring, and in particular to a method, device, equipment and medium for reconstructing a three-dimensional model of dynamic fish in a fish passage. Background Art

[0002] River damming and development have certain impacts on hydrological conditions, hydraulic characteristics, water temperature, water quality, and fish migration. According to the environmental protection requirements of water conservancy and hydropower engineering construction projects, it is necessary to establish environmental protection measures to mitigate the impact of damming, implement ecological compensation, and restore the health of river ecosystems. Fish are relatively sensitive indicator species of river ecosystems, and changes in fish populations directly reflect the status of river ecosystems. In recent years, water conservancy and hydropower projects have gradually begun to build fish-passing facilities including fishways, fish ladders, fish lifts, and fish transport vessels as connecting channels upstream and downstream of the dam. The effective operation of fish-passing facilities is an important means to protect the longitudinal connectivity of rivers, and has played an important role in mitigating the impact of dam barriers, fish migration, and promoting upstream and downstream population and gene exchanges.

[0003] Monitoring the effect of fish-passing facilities is a necessary part of the operation of fish-passing facilities. By monitoring the actual operation effect of fish-passing facilities, analyzing the movement behavior and changes of fish when passing through the facilities, and finding out the problems in the design and construction process, it can provide a basis and accumulate experience for the improvement and optimization of the functions of fish-passing facilities. When carrying out the operation effect monitoring of fish-passing facilities, some biological indicators and non-biological indicators are mainly recorded. Biological indicators include the number, species, body characteristics, age group, quality, etc. of fish passing through, and non-biological indicators include water temperature, flow, head difference, water level changes, etc. Among them, biological indicator monitoring technology is the focus and difficulty of fish-passing effect monitoring.

[0004] In recent years, with the rapid development of machine vision technology, measuring animal body size parameters through non-contact methods has become an inevitable trend. However, since fish are in an active state during the monitoring process, the visual technology images are also images of fish in various dynamic postures, which brings difficulties to the body size measurement and feature extraction of fish, and thus affects the accuracy of body size measurement and feature extraction. Summary of the invention

[0005] The purpose of the present application is to provide a method, device, equipment and medium for reconstructing a three-dimensional model of dynamic fish in a fish passage, so as to improve the accuracy of body size measurement and feature extraction.

[0006] To achieve the above objectives, this application provides the following solutions.

[0007] In a first aspect, the present application provides a method for obtaining depth image data of dynamic fish in a fish passage;

[0008] Preprocessing the depth image data to obtain preprocessed depth image data;

[0009] Perform intelligent fish segmentation on the pre-processed depth image data to obtain dynamic fish outlines;

[0010] Perform deformation correction on the dynamic fish outline image to obtain fish outline coordinate data in a straight line state and fish skeleton coordinate data in a straight line state;

[0011] Three-dimensional reconstruction is performed based on the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state to obtain a three-dimensional reconstructed model of the fish in a straight line state.

[0012] In a second aspect, the present application provides a device for reconstructing a three-dimensional model of a dynamic fish in a fish passage, wherein the device for reconstructing a three-dimensional model of a dynamic fish in a fish passage applies the above-mentioned method for reconstructing a three-dimensional model of a dynamic fish in a fish passage, and the device for reconstructing a three-dimensional model of a dynamic fish in a fish passage comprises:

[0013] A depth image acquisition module is used to acquire depth image data of dynamic fish in the fish passage;

[0014] A preprocessing module, used to preprocess the depth image data to obtain preprocessed depth image data;

[0015] The fish intelligent segmentation module is used to perform intelligent fish segmentation on the pre-processed depth image data to obtain the dynamic fish outline map;

[0016] A correction module is used to perform deformation correction on the dynamic fish outline image to obtain fish outline coordinate data in a straight line state and fish skeleton coordinate data in a straight line state;

[0017] The three-dimensional reconstruction module is used to perform three-dimensional reconstruction based on the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state to obtain a three-dimensional reconstructed model of the fish in a straight line state.

[0018] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned method for reconstructing a three-dimensional model of dynamic fish in a fish passage.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for reconstructing a three-dimensional model of dynamic fish in a fish passage.

[0020] According to the specific embodiments provided in this application, this application has the following technical effects.

[0021] The present application provides a method, device, equipment and medium for reconstructing a three-dimensional model of dynamic fish in a fish passage. The present application realizes the restoration of 1:1 real fish data based on the depth image technology, and transforms the swimming fish data into linear state data (i.e., transforms the dynamic fish into linear static state fish) based on the deformation correction method, thereby completing the three-dimensional reconstruction. Based on the three-dimensional reconstruction model of the fish, complete and accurate fish body geometric parameters can be obtained, and the fish body features can be further analyzed and extracted. The present application improves the accuracy of body size measurement and feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic flow chart of a method for reconstructing a three-dimensional model of dynamic fish in a fish passage provided in one embodiment of the present application.

[0024] Figure 2 A schematic diagram of the structure of a CNN network model provided in one embodiment of the present application.

[0025] Figure 3 A schematic diagram of the structure of a fish segmentation network model provided in one embodiment of the present application.

[0026] Figure 4 A flowchart of a deformation correction algorithm provided in one embodiment of the present application.

[0027] Figure 5 A framework diagram of fish characteristics provided in one embodiment of the present application.

[0028] Figure 6 A schematic diagram of fish body partitioning provided in one embodiment of the present application.

[0029] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0032] With the widespread use of depth cameras, it has become more convenient to obtain 3D data of objects. Therefore, a 3D reconstruction and body size measurement solution for underwater fish based on dual-view depth cameras has been developed. It can not only obtain the 3D outline of underwater dynamic fish, but also perform 1:1 3D reconstruction, thereby extracting the body size information of fish and analyzing and identifying fish characteristics through a real 3D reconstruction model.

[0033] In an exemplary embodiment, Figure 1 and Figure 2 As shown, a method for reconstructing a three-dimensional model of dynamic fish in a fish passage is provided, including the following steps 101 to 105.

[0034] Step 101, obtaining depth image data of dynamic fish in a fish passage.

[0035] Step 102: preprocess the depth image data to obtain preprocessed depth image data.

[0036] Step 103, performing intelligent fish segmentation on the pre-processed depth image data to obtain a dynamic fish outline map.

[0037] Step 104, performing deformation correction on the dynamic fish outline image to obtain fish outline coordinate data in a straight line state and fish skeleton coordinate data in a straight line state.

[0038] Step 105 , performing three-dimensional reconstruction based on the fish outline coordinate data in the straight state and the fish skeleton coordinate data in the straight state, to obtain a three-dimensional reconstructed model of the fish in the straight state.

[0039] By implementing the above steps 101 to 105, reconstruction of the three-dimensional reconstruction model of the fish in a 1:1 straight line state can be achieved, thereby improving the accuracy of body size measurement and feature extraction.

[0040] In another exemplary embodiment of the present application, in order to obtain depth image data of dynamic fish in the fish passage, a depth camera is installed on the left and right sides, upper and lower sides, and front and rear sides of the measurement channel to collect depth image data of dynamic fish in the fish passage. In this step, depth image data of dynamic fish in the fish passage in various directions (including phenotypic image data and depth information data, wherein the depth information represents the distance of the target object from the collection device) is obtained through depth cameras with three viewing angles.

[0041] In another exemplary embodiment of the present application, the above step 102 pre-processes the collected depth image data, and performs denoising, defogging, deblurring and repair operations on the depth image data collected underwater through a CNN network model. Figure 2 As shown in Figure 2, the CNN network model is based on the CHAN framework and includes a generator and a discriminator, where the generator also includes an encoding network G Enc , decoding network G Dec and feature reshaping network G Rec . The input of the generator is a distorted underwater image (i.e., the depth image data collected by the depth camera). In the generator, the depth image data is first downsampled through the encoding network, and then the deeper features are extracted through the feature reshaping network. Then, the extracted features are upsampled through the decoding network to restore the original size of the image, thereby obtaining an enhanced clear underwater image (i.e., the preprocessed depth image data). Furthermore, the encoding network extracts features through the Feature Attention Residual (FAR) module, and the decoding network refines features through the Strengthen-Operate-Subtract (SOS) module. The entire generator is supplemented by the Adaptive Dense Feature Fusion (ADFF) module for feature fusion. The discriminator includes multiple convolutional layers connected in sequence and a multi-scale fusion (MF) module, and the MF module is connected to the output end of the last convolutional layer in the multiple convolutional layers connected in sequence. The input of the discriminator is the enhanced clear underwater image and its corresponding ground truth image (GroundTruth). In the discriminator, the input image is firstly extracted through the convolution layer, and then the multi-scale feature discrimination is performed through the MF module to obtain a discrimination value to supervise the training of the generator. During the training phase, the generator and the discriminator use a cyclic alternating training method to update the gradient parameters. After the training is completed, only the trained generator is used for deep image data enhancement. Then the fish image data is extracted through the semantic segmentation algorithm.

[0042] In another exemplary embodiment of the present application, the above-mentioned step 103 pre-processes the collected depth image data, and based on the fish segmentation network model of the residual network and multi-layer parallel void convolution, performs intelligent fish segmentation on the pre-processed depth image data to obtain a dynamic fish contour map.

[0043] Figure 3 (a) is a schematic diagram of the structure of the fish segmentation network model. Figure 3 (b) is a schematic diagram of the structure of the first upsampling module in the fish segmentation network model. Figure 3As shown, the fish segmentation network model includes: a model encoder and a model decoder.

[0044] Among them, the model encoder includes: a first encoding module, a second encoding module, a third encoding module, a fourth encoding module and a fifth encoding module connected in sequence; the first encoding module includes: a 7×7 convolutional layer, a regularization layer and a Relu layer connected in sequence; the second encoding module includes: 3 residual modules; the third encoding module includes: 4 residual modules; the fourth encoding module includes: 6 residual modules; the fifth encoding module includes: 3 residual modules; the residual module includes: a 1×1 convolutional layer, a 3×3 convolutional layer, a 1×1 convolutional layer and a residual connection structure connected in sequence.

[0045] The encoding process of the model encoder is as follows: perform 7×7 convolution, regularization, and Relu on the input preprocessed depth image data to obtain the first encoding feature map, the first encoding feature map is obtained by building 3 residual modules to obtain the second encoding feature map, the second encoding feature map is obtained by building 4 residual modules to obtain the third encoding feature map, the third encoding feature map is obtained by building 6 residual modules to obtain the fourth encoding feature map, and the fourth encoding feature map is obtained by building 3 residual modules to obtain the fifth encoding feature map.

[0046] The model encoder of the present application uses a residual network as a backbone extraction network to extract target semantic information, which can effectively avoid gradient vanishing and gradient exploding while deeply extracting features.

[0047] The model decoder includes: a plurality of first upsampling modules and a second upsampling module connected in sequence, wherein the second upsampling module is connected to the last first upsampling module among the plurality of first upsampling modules connected in sequence; the first upsampling module includes a splicing and fusion layer, a 3×3 convolution layer, a 3×3 convolution layer and a multi-layer parallel hole convolution layer connected in sequence; the second upsampling module includes a splicing and fusion layer, a 3×3 convolution layer, a 3×3 convolution layer, a multi-layer parallel hole convolution layer and a 1×1 convolution layer connected in sequence.

[0048] The decoding process of the model decoder is as follows: for the feature map obtained by the model encoder, the high-level semantic information feature map generated by upsampling and the shallow semantic information feature map generated in the corresponding stage of the first half are spliced ​​and fused through jump connections. The fused decoded feature map is convoluted twice with 3×3 convolutions and then subjected to a multi-layer parallel dilated convolution. Convolution and pooling operations are performed at four sampling rates of 1, 6, 12, and 18 to expand the receptive field and supplement the missing information after the fish edge convolution. Finally, a clear and accurate fish outline map is obtained through a classifier.

[0049] In another exemplary embodiment of the present application, in the above step 104, edge detection and skeleton extraction are performed by Hough transform algorithm to obtain dynamic fish outline coordinate data and dynamic fish skeleton coordinate data, and the fish outline image is corrected at the same time, the direction information of the skeleton is determined, and the cutting points required for correction are determined, and finally the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state are obtained by image cutting, rotation and synthesis. Specifically, this step can be replaced by the following steps 201-204.

[0050] Step 201, use an edge detection algorithm to perform edge detection on the dynamic fish contour image to obtain dynamic fish contour coordinate data and dynamic fish skeleton coordinate data; the dynamic fish contour coordinate data includes multiple dynamic fish contour coordinates, and the dynamic fish skeleton coordinate data includes multiple dynamic fish skeleton coordinates.

[0051] Step 202, constructing a center straight line based on the dynamic fish skeleton coordinate data; the number of dynamic fish skeleton coordinate points in the dynamic fish skeleton coordinate data located on the center straight line is greater than a preset threshold.

[0052] Step 203, determining the deviation distance of each dynamic fish skeleton coordinate point relative to the central straight line.

[0053] Step 204, based on the deviation distance of each dynamic fish skeleton coordinate point relative to the central straight line, each dynamic fish skeleton coordinate point and each dynamic fish contour coordinate corresponding to each dynamic fish skeleton coordinate point are offset to obtain fish contour coordinate data in a straight line state and fish skeleton coordinate data in a straight line state.

[0054] like Figure 4 As shown, the specific algorithm description of the above step 104 is as follows:

[0055] Image preprocessing: Grayscale, converting color images into grayscale images to simplify the processing process.

[0056] Edge Detection: Use the Canny edge detector or other edge detection algorithms to identify edges in the image.

[0057] Parameter space initialization: Define the parameter space, ρ (the vertical distance from the origin to the line) and θ (the angle between the line and the x-axis).

[0058] Accumulator array creation: Set the accumulator size, determine the range of ρ and θ based on the required accuracy, and set the size of the accumulator array accordingly.

[0059] Mapping to Hough space: Map each edge point. For each edge point (x, y) in the image, calculate all the possible corresponding line parameters (ρ, θ). For each possible (ρ, θ), increase the count at the corresponding position in the accumulator array.

[0060] Finding the peak: Set a threshold, and only the (ρ, θ) pairs whose accumulator values ​​exceed the threshold are considered valid line parameters. Find the local maximum in the accumulator array to avoid detecting multiple overlapping lines.

[0061] Extract the equation of the line: For each peak position, obtain its ρ and θ values. Use the ρ and θ values ​​to calculate the standard equation or polar coordinate equation of the line.

[0062] Result visualization: Draw the detected lines on the original image or output them to a new image.

[0063] In another exemplary embodiment of the present application, in the above step 105, three-dimensional modeling of the fish is performed through the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state to obtain a three-dimensional reconstructed model of the fish, and point cloud computing, point cloud registration, and data fusion are performed through the fish outline coordinate data in a straight line state and the fish skeleton coordinate data obtained in step 104 to finally obtain a fine three-dimensional reconstructed model of the fish in a straight line state and output the coordinate data of the three-dimensional reconstructed model of the fish.

[0064] The specific steps are as follows:

[0065] Data preprocessing: remove noise points in the point cloud data and align the point cloud data acquired from different perspectives or at different times.

[0066] Data simplification: Reduce the amount of point cloud data to reduce the computational complexity of subsequent processing while maintaining the geometric features of the model as much as possible. Convert point cloud data into a triangular mesh model to facilitate subsequent modeling and visualization.

[0067] Feature extraction and surface reconstruction: Calculating the normal direction of each point helps to understand the geometric features of the surface. Use surface reconstruction algorithms (such as Poisson reconstruction, implicit surface reconstruction, etc.) to convert point cloud data into a continuous surface model.

[0068] Model optimization: Analyze the errors of the reconstructed model and make necessary adjustments and optimizations. Increase the details of the model as needed to make it closer to the real object.

[0069] Model export: Export the final model to common 3D file formats.

[0070] In another exemplary embodiment of the present application, after the above step 105, a step of calculating the body size parameters and extracting characteristic indicators of the fish based on the three-dimensional reconstruction model of the fish in a straight line state is further performed.

[0071] Body size parameters: The key body size parameters of fish are calculated through the coordinate data in the three-dimensional reconstruction model of the fish in a straight line state. The body size parameters include body length, body width and body height. Among them, the body length is calculated by the distance between the coordinates at the fish lips and the coordinates at the fish tail, the body width is calculated by the distance between the coordinates on both sides of the fish, and the body height is calculated by the distance between the coordinates of the highest and lowest points of the fish.

[0072] Feature extraction: shape and texture features are extracted through the 3D reconstruction model of the fish in a straight line state, such as Figure 5 Shown and Figure 6 As shown in the figure, morphological features such as the number, position and shape of dorsal fin, caudal fin, pectoral and pelvic fins, and tail fin, fish eyes and spine outlines can be directly extracted from the image, and the above features are classified as the primary indicators of fish phenotypic characteristics. Features that require secondary calculation such as the ratio of the major axis to the minor axis, tail shape characteristics, fin shape characteristics, relative position characteristics, etc. can be summarized as the secondary indicators of fish phenotypic characteristics. A fish phenotypic characteristic database is established through the obtained primary and secondary indicator parameters.

[0073] According to the specific embodiments provided in this application, this application has the following technical effects.

[0074] The depth camera is used to obtain the depth image data of the dynamic fish in the fish passage in three directions, and the image data is restored to the real fish data in a 1:1 ratio according to the depth image data.

[0075] Through the deformation correction algorithm, the swimming fish is transformed into a static state and the complete fish body geometric parameters are obtained, from which the fish body characteristics can be further analyzed and extracted.

[0076] Based on the same inventive concept, the embodiment of the present application also provides a device for reconstructing a three-dimensional model of dynamic fish in a fish passage for implementing the three-dimensional model reconstruction method of dynamic fish in a fish passage as mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more devices for reconstructing a three-dimensional model of dynamic fish in a fish passage provided below can refer to the limitations of the method for reconstructing a three-dimensional model of dynamic fish in a fish passage above, and will not be repeated here.

[0077] In an exemplary embodiment, a device for reconstructing a three-dimensional model of dynamic fish in a fish passage is provided, comprising:

[0078] The depth image acquisition module is used to acquire the depth image data of dynamic fish in the fish passage.

[0079] The preprocessing module is used to preprocess the depth image data to obtain preprocessed depth image data.

[0080] The fish intelligent segmentation module is used to perform intelligent fish segmentation on the pre-processed depth image data to obtain the dynamic fish outline map.

[0081] The correction module is used to perform deformation correction on the dynamic fish outline image to obtain the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state.

[0082] The three-dimensional reconstruction module is used to perform three-dimensional reconstruction based on the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state to obtain a three-dimensional reconstructed model of the fish in a straight line state.

[0083] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for reconstruction and measurement results. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a three-dimensional model reconstruction method of dynamic fish in a fish passage is implemented.

[0084] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0085] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0087] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0088] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0089] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for reconstructing a three-dimensional model of dynamic fish in a fish passage, characterized in that: include: Obtain depth image data of dynamic fish in the fish passage; Preprocessing the depth image data to obtain preprocessed depth image data; Perform intelligent fish segmentation on the pre-processed depth image data to obtain dynamic fish outlines; Perform deformation correction on the dynamic fish outline image to obtain fish outline coordinate data in a straight line state and fish skeleton coordinate data in a straight line state; Three-dimensional reconstruction is performed based on the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state to obtain a three-dimensional reconstructed model of the fish in a straight line state.

2. The method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to claim 1, characterized in that: Preprocessing the depth image data to obtain preprocessed depth image data specifically includes: The generator is used to downsample, extract deep features and upsample the depth image data to obtain the preprocessed depth image data.

3. The method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to claim 2, characterized in that: The generator includes an encoding network, a feature reshaping network and a decoding network connected in sequence; The encoding network, the feature reshaping network and the decoding network are used for downsampling, deep-level feature extraction and upsampling respectively.

4. The method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to claim 2, characterized in that: The generator is obtained by performing cyclic alternating training with the discriminator; The discriminator includes a plurality of convolutional layers and an MF module connected in sequence, and the MF module is connected to the output end of the last convolutional layer among the plurality of convolutional layers connected in sequence.

5. The method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to claim 1, characterized in that: The pre-processed depth image data is intelligently segmented to obtain the dynamic fish outline map, which includes: The fish segmentation network model is used to intelligently segment the preprocessed depth image data to obtain the dynamic fish outline map; The fish segmentation network model comprises: a model encoder and a model decoder; The model encoder comprises: a first encoding module, a second encoding module, a third encoding module, a fourth encoding module and a fifth encoding module connected in sequence; The first encoding module includes: a 7×7 convolutional layer, a regularization layer and a ReLU layer connected in sequence; The second encoding module includes: 3 residual modules; The third encoding module includes: 4 residual modules; The fourth encoding module includes: 6 residual modules; The fifth encoding module includes: 3 residual modules; The residual module includes: a 1×1 convolutional layer, a 3×3 convolutional layer, a 1×1 convolutional layer and a residual connection structure connected in sequence; The model decoder comprises: a plurality of first up-sampling modules and a second up-sampling module connected in sequence, wherein the second up-sampling module is connected to the last first up-sampling module among the plurality of first up-sampling modules connected in sequence; The first upsampling module includes a splicing fusion layer, a 3×3 convolution layer, a 3×3 convolution layer, and a multi-layer parallel hole convolution layer connected in sequence; The second upsampling module includes a splicing fusion layer, a 3×3 convolution layer, a 3×3 convolution layer, a multi-layer parallel hole convolution layer and a 1×1 convolution layer, which are connected in sequence.

6. The method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to claim 1, characterized in that: The deformation correction is performed on the dynamic fish outline image to obtain the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state, specifically including: An edge detection algorithm is used to perform edge detection on a dynamic fish outline image to obtain dynamic fish outline coordinate data and dynamic fish skeleton coordinate data; the dynamic fish outline coordinate data includes a plurality of dynamic fish outline coordinates, and the dynamic fish skeleton coordinate data includes a plurality of dynamic fish skeleton coordinates; Constructing a center straight line based on the dynamic fish skeleton coordinate data; the number of dynamic fish skeleton coordinate points located on the center straight line in the dynamic fish skeleton coordinate data is greater than a preset threshold; Determine the deviation distance of each dynamic fish skeleton coordinate point relative to the central straight line; Based on the deviation distance of each dynamic fish skeleton coordinate point relative to the center straight line, each dynamic fish skeleton coordinate point and each dynamic fish contour coordinate corresponding to each dynamic fish skeleton coordinate point are offset to obtain fish contour coordinate data in a straight line state and fish skeleton coordinate data in a straight line state.

7. The method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to claim 1, characterized in that: Based on the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state, a three-dimensional reconstruction model of the fish in a straight line state is obtained, and then the following is also included: The body size parameters of the fish are calculated and characteristic indicators are extracted based on the three-dimensional reconstruction model of the fish in a straight line state; the body size parameters include body length, body width and body height; the characteristic indicators include: primary indicators and secondary indicators, the primary indicators include color, shape and texture, and the secondary indicators include: major-minor axis ratio, tail shape characteristics, fin type and relative position.

8. A device for reconstructing a three-dimensional model of dynamic fish in a fish passage, characterized in that: The device for reconstructing the three-dimensional model of dynamic fish in the fish passage adopts the method for reconstructing the three-dimensional model of dynamic fish in the fish passage according to any one of claims 1 to 7, and the device for reconstructing the three-dimensional model of dynamic fish in the fish passage comprises: A depth image acquisition module is used to acquire depth image data of dynamic fish in the fish passage; A preprocessing module, used to preprocess the depth image data to obtain preprocessed depth image data; The fish intelligent segmentation module is used to perform intelligent fish segmentation on the pre-processed depth image data to obtain the dynamic fish outline map; A correction module is used to perform deformation correction on the dynamic fish outline image to obtain fish outline coordinate data in a straight line state and fish skeleton coordinate data in a straight line state; The three-dimensional reconstruction module is used to perform three-dimensional reconstruction based on the fish outline coordinate data in a straight line state and the fish skeleton coordinate data in a straight line state to obtain a three-dimensional reconstructed model of the fish in a straight line state.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for reconstructing a three-dimensional model of dynamic fish in a fish passage according to any one of claims 1 to 7 is implemented.

Images