A method, device, equipment and medium for characterizing and measuring underwater leopard-blotched wrasse

Abstract

This invention discloses an underwater characterization measurement method, apparatus, device, and medium for leopard-gill perch. The method includes: acquiring underwater images of the leopard-gill perch using a binocular camera; detecting key points of the leopard-gill perch using a key point detection model to obtain the pixel positions of each key point in the underwater image; establishing an underwater light refraction model; performing refraction offset compensation on the pixel positions of the key points in the underwater image using the underwater light refraction model to convert the pixel positions of the key points in the underwater image into equivalent pixel positions corresponding to the air medium; acquiring the spatial positions of each key point using stereo vision technology; and determining the characterization data of the underwater leopard-gill perch based on the spatial position data of each key point. This invention improves the accuracy of key point detection by performing refraction offset compensation on the pixel coordinates of the key points of the leopard-gill perch in the underwater image using an underwater light refraction model, enabling characterization measurement with millimeter-level precision.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method, apparatus, computer device, and computer-readable storage medium for characterizing and measuring underwater leopard gills. Background Technology

[0002] Leopard spiny perch (Sebastes nebulosus) is an important marine economic fish species widely used in aquaculture. With the development of intelligent aquaculture, accurate measurement of underwater organism characteristics (such as body length, weight, and gill spines) is of great significance for assessing the aquaculture environment, optimizing production efficiency, and monitoring fish health.

[0003] Traditional manual measurement is typically inefficient and time-consuming, especially in large-scale aquaculture, making it difficult to meet the demands for rapid and efficient data collection. Measurement results are easily affected by operator skill and the underwater environment, leading to significant errors. Furthermore, accuracy is difficult to guarantee due to limitations such as lighting and transparency. In addition, manual measurement suffers from strong subjectivity and poor consistency, making it difficult to provide stable and reliable results in practical applications. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, device, and medium for characterizing and measuring underwater leopard-gill sea bass, which accurately calculates the characterization data of underwater leopard-gill sea bass using underwater images of the sea bass, thus solving the accuracy and efficiency problems in traditional measurement methods.

[0005] The first aspect of the present invention provides a method for characterizing and measuring underwater leopard-gill spiny perch, the method comprising:

[0006] Underwater images of the leopard gill sea bass were acquired using a binocular camera, and key points of the leopard gill sea bass were detected using a key point detection model to obtain the pixel position of each key point in the underwater image.

[0007] Establish an underwater light refraction model;

[0008] By using an underwater light refraction model, the pixel positions of key points in the underwater image are refraction offset compensation, converting the pixel positions of key points in the underwater image into equivalent pixel positions in the air medium.

[0009] By combining the equivalent pixel positions of key points in the air medium and the internal and external parameters of the binocular camera, the spatial position of each key point is obtained through stereo vision technology.

[0010] Based on the spatial location data of each key point, characterization data of the underwater leopard gill spiny perch were determined.

[0011] In some embodiments, the method further includes:

[0012] Underwater images of the leopard gill perch were acquired using a binocular camera. First, the leopard gill perch were identified and its region was detected using a target detection model. Then, key points of the leopard gill perch were detected using a key point detection model, and the pixel positions of each key point in the underwater image were obtained.

[0013] In some embodiments, key features of the leopard gill spiny perch include: snout, eyes, upper caudal fin, lower caudal fin, anterior dorsal fin, posterior dorsal fin, upper dorsal fin, pelvic fin, and anal fin.

[0014] In some embodiments, an underwater light refraction model is established, including:

[0015]

[0016] In the formula, x r x represents the pixel location of the key point in the underwater image. a The key point corresponds to the equivalent pixel position in the air medium; T is the thickness of the waterproof cover of the stereo camera, d is the distance between the waterproof cover and the stereo camera; n r Let n be the refractive index of water. g n is the refractive index of the waterproof cover. a α is the refractive index of air; α is the angle of incidence.

[0017] In some embodiments, refraction offset compensation is performed on the pixel positions of key points in the underwater image using an underwater light refraction model, converting the pixel positions of key points in the underwater image into equivalent pixel positions corresponding to those in the air medium, including:

[0018]

[0019] Using the above equations, the equivalent horizontal and vertical pixel coordinates of the key points in the air medium are calculated based on the horizontal and vertical pixel coordinates of the key points in the underwater image.

[0020] In some embodiments, the spatial position of each key point is obtained using stereo vision technology by combining the equivalent pixel position of the key point in the air medium and the intrinsic and extrinsic parameters of the binocular camera, including:

[0021] Establish a mapping relationship between spatial points and image points. Define any point B in space, and let its projection points on the left and right camera image planes be B1 and B2, respectively. l and B r The corresponding pixel coordinates are (u1, v1, 1) and (u2, v2, 1), respectively. The relationship between their homogeneous coordinates and the coordinates (X, Y, Z, 1) of spatial point B is described by the following formula:

[0022]

[0023]

[0024] Where, matrix M k element m k ij The parameters represent the projection matrix of the cameras, where k = 1 and 2 represent the left and right cameras, respectively; Z c1 and Z c2 These represent the depth parameters of the left and right cameras, respectively;

[0025] Transforming the above equations algebraically, we obtain a set of linear equations for X, Y, and Z:

[0026]

[0027] The coordinates (X, Y, Z) of spatial point B are obtained by solving these two linear equations simultaneously. Then, the spatial position of each key point is obtained by the equivalent pixel position of the key point in the air medium and the intrinsic and extrinsic parameters of the binocular camera.

[0028] In some embodiments, characterization data for the underwater leopard gill spiny perch include body length, weight, and gill spine length.

[0029] According to a second aspect of the present invention, an underwater characterization and measurement device for leopard-gill perch is provided, the device comprising:

[0030] The underwater pixel location unit is used to acquire underwater images of the leopard gill sea bass through a binocular camera, and to detect key points of the leopard gill sea bass through a key point detection model to obtain the pixel location of each key point in the underwater image.

[0031] The light refraction model unit is used to build an underwater light refraction model;

[0032] The refraction offset compensation unit is used to perform refraction offset compensation on the pixel position of key points in the underwater image through the underwater light refraction model, and convert the pixel position of key points in the underwater image into the equivalent pixel position in the air medium.

[0033] The spatial position acquisition unit is used to combine the equivalent pixel position of the key point in the air medium and the internal and external parameters of the binocular camera to acquire the spatial position of each key point through stereo vision technology.

[0034] The characterization data determination unit is used to determine the characterization data of the underwater leopard gill spiny perch based on the spatial location data of each key point.

[0035] According to a third aspect of the present invention, a computer device is provided, comprising: a processor and a memory, the memory storing a program or instructions executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the underwater leopard gill spiny perch characterization measurement method as described in any one of the first aspects.

[0036] According to a fourth aspect of the invention, a readable storage medium is provided having a program or instructions stored thereon, which, when executed by a processor, implement the steps of the underwater leopard gill spiny perch characterization measurement method as described in any one of the first aspects.

[0037] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0038] (1) By establishing an accurate underwater light refraction model, the coordinates of key points of the leopard gill spiny perch in the underwater image can be compensated for refraction offset, which effectively reduces the position offset error caused by refraction of water, waterproof cover and air interface, improves the accuracy of key point detection and realizes millimeter-level precision characterization measurement.

[0039] (2) By adopting the target detection and key point detection model, the nine key point locations of the leopard gill spiny perch can be automatically identified, reducing human intervention, improving the automation level and work efficiency of measurement, and is suitable for the measurement and analysis of large-scale samples.

[0040] (3) By combining the 2D key point data after underwater refraction compensation with the internal and external parameters of the camera, the depth information in 3D space is obtained by using a stereo method, which realizes accurate 2D-3D coordinate mapping and further enhances the accuracy of characterization measurement.

[0041] (4) Compared with traditional underwater measurement methods, this invention does not require expensive special equipment. High-precision measurement can be achieved by using ordinary underwater camera systems, which reduces equipment costs and technical barriers. Attached Figure Description

[0042] Figure 1 A schematic flowchart illustrating an underwater characterization and measurement method for leopard-gill sea bass provided in this application embodiment;

[0043] Figure 2 A schematic diagram of an underwater light refraction model provided in this application embodiment;

[0044] Figure 3 A schematic diagram illustrating the mapping relationship between 3D spatial points and 2D image points, provided for an embodiment of this application;

[0045] Figure 4 A diagram showing the calculated body length of an underwater leopard-gill spiny perch, provided as an embodiment of this application;

[0046] Figure 5 A frame diagram of an underwater characterization and measurement device for leopard-gill sea bass provided in this application embodiment;

[0047] Figure 6 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this invention.

[0049] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0050] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0051] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0052] This application provides a characterization measurement method for leopard gill sea bass based on the combination of underwater light refraction and coordinate mapping, which relates to the fields of computer vision and intelligent aquaculture. Specifically, it relates to an underwater light refraction model and a 2D-3D coordinate mapping method of coordinate system matrix transformation. The aim is to accurately calculate the characterization data of underwater leopard gill sea bass through a binocular stereo vision system, thereby solving the accuracy and efficiency problems in traditional measurement methods.

[0053] This application presents a characterization measurement method for the leopard-gill sea bass based on a combination of underwater light refraction and coordinate mapping. It proposes an underwater light refraction model and a 2D-3D coordinate mapping method using matrix transformation. Pixel offset compensation is applied to key points of the leopard-gill sea bass in the image, adjusting them to their pixel positions in an equivalent atmospheric image. Combining the intrinsic and extrinsic parameters of a binocular camera and the 2D-3D coordinate mapping method, nine key points of the leopard-gill sea bass are mapped from 2D coordinates to 3D, and their characterization data are accurately calculated. This method can obtain relatively accurate characterization data using only a low-cost binocular camera, with an error at the millimeter level.

[0054] The leopard-gill perch characterization measurement method based on underwater light refraction and coordinate mapping in this application embodiment takes as input a sequence of images of the leopard-gill perch acquired through a binocular camera, and outputs as 3D characterization data of the leopard-gill perch, such as... Figure 1 As shown, the steps are as follows:

[0055] S101. Using the collected training sample image set of the leopard-gill sea bass, models for object detection and keypoint detection of the leopard-gill sea bass are trained respectively. After training, these network models will be used to automatically detect keypoints of the leopard-gill sea bass.

[0056] S102. Based on underwater optical characteristics, an underwater light refraction model is established. This model allows for refraction offset compensation of the 2D pixel coordinates of key points in underwater images of the leopard gill sea bass.

[0057] S103, by compensating for the pixel offset of the key points of the leopard gill sea bass in the pixel image, it converts them into equivalent pixel positions on the water surface or in the air. This step utilizes an optical path refraction model to improve the correspondence between the coordinates of key points in the underwater image and their coordinates in the air, eliminating the deviation caused by the waterproof cover and water refraction.

[0058] S104 combines offset-compensated 2D keypoints with the camera's intrinsic and extrinsic parameters to acquire depth information for each keypoint using stereo vision technology. Utilizing a 2D-3D coordinate mapping method, the offset-compensated 2D keypoint coordinates are mapped to 3D space, accurately obtaining the positions of the nine keypoints of the leopard gill spiny perch in 3D space.

[0059] S105, using the 3D coordinate data obtained through mapping, further calculate the characterization data of the leopard-spined sea bass, such as body length, weight, and gill spine length. Through precise 2D-3D coordinate mapping and key point calculation, this invention can achieve characterization measurements with millimeter-level accuracy.

[0060] This invention combines underwater light refraction compensation with binocular stereo vision technology to accurately extract key points from underwater images of the leopard-gill sea bass and map them from 2D image coordinates to 3D space. By modeling underwater optical characteristics and compensating for pixel offsets, errors caused by the underwater environment are eliminated, improving measurement accuracy. Combining depth information from a binocular camera with a 2D-3D coordinate mapping method, precise measurement of underwater leopard-gill sea bass characteristics is successfully achieved, providing the aquaculture industry with an efficient and accurate data acquisition technology.

[0061] In some embodiments, step S101 is specifically performed as follows:

[0062] Assuming the input image is I, the output of the object detection model is a series of bounding boxes and their corresponding class labels y. The goal is to train the model by minimizing the following loss function:

[0063]

[0064] In the formula, L cls It is a classification loss, Lreg It is the regression loss, b i and λ represents the coordinates of the ground truth bounding box and the predicted bounding box, respectively, and λ is the balance coefficient. After training, the model can automatically detect the region of the leopard gill sea bass.

[0065] Assuming image I is the input, the model output is a set of keypoint coordinates (x1, y1), (x2, y2), ..., (x... n ,y n ), where n is the total number of keypoints. The keypoint detection model is trained by minimizing the keypoint prediction error. The loss function typically uses the mean squared error:

[0066]

[0067] This application specifies nine key points of the leopard-gill spiny perch (also known as the leopard-gill spiny perch): snout, eye, upper caudal fin, lower caudal fin, anterior dorsal fin, posterior dorsal fin, upper dorsal fin, pelvic fin, and anal fin. (x i ,y i )and These are the actual and predicted coordinates of keypoints in nine different regions. For each image in the training set, the model learns the optimal predicted location for each keypoint by minimizing this loss function.

[0068] In some embodiments, step S102 is specifically performed as follows:

[0069] Based on optical principles, and combining the refractive index and the refraction effect of light passing through different media (such as air, waterproof casing, and water), a light refraction model is established. This model is used to simulate the propagation path of light in different media to calculate the pixel coordinate offset caused by refraction.

[0070] like Figure 2 As shown, the system is projected onto a plane, considering only the lateral distance z and longitudinal distance x of the eastern star spot from the imaging plane (setting the y-axis coordinate to 0). A coordinate system is defined, assuming the target point p... w (x w ,0,z w The projection point p is located in the water. a (x a ,0) and p r (x r (x, 0) represent the image positions after refraction, respectively. On the image plane, x... r and x a This represents the projected position of the target in air and water.

[0071] The parameters are defined as follows: T is the thickness of the waterproof cover, Z is the distance from the target point to the camera, d is the distance between the waterproof cover and the camera, and n...a ,n g ,n r These represent the refractive indices of air, the waterproof cover, and water, respectively.

[0072] The relationship of light refraction is established based on the law of refraction, which states that light rays between different media will refract at the interface. Let the angle of incidence and the angle of refraction be:

[0073] Air / water-waterproof cover interface: angle of incidence α, angle of refraction of the object in air α g,a The angle of refraction α of an object in water g,r ;

[0074] Waterproof cover - air / water interface: angle of incidence α of the object in air g,a The angle of incidence α of the object in the water g,r , angle of refraction α r .

[0075] The refraction relationship is as follows:

[0076] n r sinα=n g ·sinα g,r =n a ·sinα r

[0077] n a sinα=n g ·sinα g,a =n a ·sinα a

[0078] The derivation of the refraction model formula, from Figure 2 The following coordinate relationships can be derived from this. Through geometric relationships and the law of refraction, the projected coordinates x on the image plane are obtained. r and x a :

[0079]

[0080] The refraction offset compensation is achieved using the above formula, which is used to locate the key point p in the underwater image. w (x w ,0,z w After refraction and transformation at the air-waterproof cover-water interface, the equivalent image position p corresponding to the air medium is obtained. a (x a ,0). This position is calculated using the formula above, thus eliminating image position deviation caused by underwater refraction.

[0081] It should be noted that the above example is based on the x-direction; the same principle applies to the y-direction.

[0082] In some embodiments, step S103 specifically involves the following steps:

[0083] The application of the refraction model, based on the principle of light path refraction, sets the imaging coordinates of key points in the underwater image as p. r (x r The corresponding imaging coordinates in the air are p, 0). a (x a Based on the formula derived in the second step, the following equation can be established:

[0084]

[0085] Let x r For any keypoint location detected by the model in an underwater image, its equivalent air keypoint location x can be calculated using the above equation. a Applying this equation to all key points of the leopard gill sea bass ensures accurate location and measurement of the sea bass in underwater environments.

[0086] It should be noted that the equivalent horizontal pixel coordinates of the key point in the air medium can be calculated by using the horizontal pixel coordinates of the key point in the underwater image and the corresponding incident angle. Similarly, the equivalent vertical pixel coordinates of the key point in the air medium can be calculated by using the vertical pixel coordinates of the key point in the underwater image and the corresponding incident angle. Thus, the equivalent pixel position of the key point in the air medium can be obtained.

[0087] In some embodiments, step S104 is specifically performed as follows:

[0088] Establish a mapping relationship between spatial points and image points. Define any point B in space, and let its projection points on the left and right camera image planes be B1 and B2, respectively. l and B r ,like Figure 3 As shown. The pixel coordinates of the left and right cameras in the image are (u1, v1, 1) and (u2, v2, 1), respectively. The relationship between their homogeneous coordinates and the coordinates (X, Y, Z, 1) of the spatial point B can be described by the following formula:

[0089]

[0090] Where, matrix M k element m k ij The parameter represents the projection matrix of the camera, where k = 1 and 2 represent the left and right cameras, respectively.

[0091] Eliminate depth parameter Z c1 and Z c2 In order to eliminate the depth parameter Zc1 and Z c2 The above equations can be transformed algebraically to obtain a set of linear equations for X, Y, and Z:

[0092]

[0093] These two sets of equations are derived from observations from the left and right cameras, respectively. The coordinates (X, Y, Z) of spatial point B can be solved by simultaneously solving these two sets of equations. In practical applications, to reduce errors caused by image noise, the least squares method can be used to solve the above linear equations to obtain the best estimate of spatial point B (X, Y, Z).

[0094] In some embodiments, step S105 specifically involves the following steps:

[0095] Applying 2D-3D mapping to keypoints, this matrix transformation method can convert the keypoint coordinates (u1, v1) and (u2, v2) of the leopard gill sea bass detected in the image into their actual position coordinates in 3D space. For keypoints of the leopard gill sea bass, such as the head and tail, their 3D coordinates H(X) can be calculated separately. H ,Y H Z H ) and T(X T ,Y T Z T ).

[0096] Based on the calculated 3D coordinates of key points, characterizing data such as body length can be further calculated for the Leopard Gill Spinach, such as... Figure 4 As shown. For example, body length can be calculated using the Euclidean distance between the head and tail key points:

[0097]

[0098] This allows for accurate measurement of the body length and other parameters of the leopard-gill spiny perch, thus achieving high-precision characterization measurements.

[0099] Specifically, the experimental platform in this embodiment is equipped with an NVIDIA A100-PCIE-40GB graphics card, providing powerful computing capabilities. The experimental code environment is based on Python 3.10, primarily using a deep learning framework for model training and keypoint detection. The dataset consists of 400 images of the leopard-gill sea bass that we independently captured, covering different poses and lighting conditions to ensure the robustness and accuracy of the model. By training the target detection and keypoint detection model for the leopard-gill sea bass, automatic detection and localization of keypoints can be achieved, thus laying the foundation for subsequent characterization measurements.

[0100] The process of this embodiment is shown below. Figure 1 As shown, the method includes the following steps:

[0101] S101: Using the collected training sample image set of leopard gill sea bass, train the target detection and key point detection models for leopard gill sea bass respectively.

[0102] (1) In the Python 3.10 environment, import the training sample image set of the leopard gill sea bass. The image sequence is denoted as I. i (x), each image contains multiple Leopard Gill Spinach targets in different poses:

[0103] (2) Use data augmentation techniques such as rotation, scaling, and cropping to expand the training samples to improve the model's generalization ability and robustness:

[0104] (3) Through batch training and parameter optimization, ensure that the target detection model and key point detection model achieve the expected accuracy and stability.

[0105] S102: Based on underwater optical properties, establish an underwater light refraction model.

[0106] (1) For each key point in the image of the leopard gill sea bass, transform from the camera coordinate system to the world coordinate system of the underwater environment to determine the direction of light propagation. Calculate the angle of refraction of the light at the water-air interface according to Snell's law;

[0107] (2) A refraction model of underwater light is constructed using the refractive indices of water and air. This model can accurately describe the changes in the direction of light propagation in different media, so that the coordinates of 2D key points in underwater images can be mapped to their equivalent positions in the air medium.

[0108] S103: By compensating for the pixel offset of the key point of the leopard gill perch in the pixel image, it is converted into an equivalent pixel position on the water surface or in the air.

[0109] (1) Based on the change in refraction angle along the light propagation path, the underwater pixel coordinates of each key point are compensated for refraction offset. For the offset-compensated key point coordinates, a mathematical transformation is performed to ensure that their positions in the air match the corresponding points in the underwater image;

[0110] (2) Map the coordinates of the key points after refraction offset compensation to the equivalent air medium coordinates to eliminate optical errors caused by the different media of water and waterproof cover.

[0111] S104: Combining the offset-compensated 2D key points with the camera's intrinsic and extrinsic parameters, the 2D-3D coordinate mapping method is used to map the offset-compensated 2D key point coordinates to 3D space.

[0112] (1) Using the offset-compensated 2D keypoint coordinates, and combining them with the camera's intrinsic and extrinsic parameters, determine the keypoint positions in the offset-compensated image. These parameters include the camera's focal length, optical center position, and perspective distortion coefficient;

[0113] (2) Combine the offset compensation 2D coordinates of each key point with the corresponding depth information, and use the 2D-3D mapping algorithm to transform them into the 3D coordinate system to ensure that the nine key points of the leopard gill spiny perch are accurately located in 3D space.

[0114] (3) Based on the initial depth estimation, the depth value of each key point is adjusted through multiple optimization iterations to achieve higher accuracy, so that the 3D coordinates are more consistent with the key point positions in real space.

[0115] S105: Utilize 3D coordinate data for precise characterization and measurement.

[0116] (1) Based on the 3D keypoint coordinates obtained by mapping, multiple characterization data of the leopard gill spiny perch were calculated. These data include specific measurement indicators such as body length, weight, and gill spine length;

[0117] (2) Output the final characterization measurement data to form a high-precision characterization data set that meets the application requirements for further analysis or application.

[0118] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0119] This application also provides an underwater characterization and measurement device for leopard-gill spiny perch. These devices are used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] Figure 5 This is a structural block diagram of the underwater leopard-gill spiny perch characterization and measurement device according to an embodiment of this application, as shown below. Figure 5 As shown, the device includes an underwater pixel position unit 201, a light refraction model unit 202, a refraction offset compensation unit 203, a spatial position acquisition unit 204, and a characterization data determination unit 205.

[0121] The underwater pixel location unit 201 is used to acquire underwater images of the leopard gill sea bass through a binocular camera, and detect key points of the leopard gill sea bass through a key point detection model to obtain the pixel location of each key point in the underwater image.

[0122] Light refraction model unit 202 is used to establish an underwater light refraction model;

[0123] The refraction offset compensation unit 203 is used to perform refraction offset compensation on the pixel position of key points in the underwater image through the underwater light refraction model, and convert the pixel position of key points in the underwater image into the equivalent pixel position in the air medium.

[0124] The spatial position acquisition unit 204 is used to combine the equivalent pixel position of the key point in the air medium and the internal and external parameters of the binocular camera to acquire the spatial position of each key point through stereo vision technology.

[0125] Characterization data determination unit 205 is used to determine the characterization data of underwater leopard gill spiny perch based on the spatial location data of each key point.

[0126] It should be noted that the aforementioned units can be either functional units or program units, and can be implemented in either software or hardware. For units implemented in hardware, the aforementioned units can reside in the same processor; or the aforementioned units can be located in different processors in any combination.

[0127] In addition, combined Figure 1 The underwater leopard gill spiny perch characterization measurement method described in this application embodiment can be implemented by a computer device. Figure 6 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application. Figure 6 As shown, the device may include a processor 301 and a memory 302 storing computer program instructions.

[0128] Specifically, the processor 301 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0129] Memory 302 may include a mass storage device for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to a data processing device. In a particular embodiment, memory 302 is non-volatile memory. In a particular embodiment, memory 302 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only ROM (EPROM), an electrically erasable read-only ROM (EEPROM), an electrically alterable read-only ROM (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0130] The memory 302 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 301.

[0131] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any of the underwater leopard gill spiny perch characterization and measurement methods in the above embodiments.

[0132] In some embodiments, the point cloud generation device may further include a communication interface 303 and a bus 300. Wherein, as... Figure 6 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 300 and complete communication with each other.

[0133] The communication interface 303 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication interface 303 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0134] Bus 300 includes hardware, software, or both, that couples the components of the point cloud generation device together. Bus 300 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 300 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 300 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnection.

[0135] The computer device can execute the underwater leopard gill spiny perch characterization measurement method of the embodiments of this application based on the rendering device, thereby achieving a combination of Figure 1 The underwater characterization and measurement method for the leopard-gill spiny perch is described.

[0136] Furthermore, in conjunction with the underwater leopard-gill spiny perch characterization measurement method in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any one of the underwater leopard-gill spiny perch characterization measurement methods in the above embodiments.

[0137] In summary, this application proposes a method, apparatus, equipment, and medium for underwater characterization and measurement of leopard-gill sea bass based on stereo vision and refraction offset compensation. The aim is to acquire high-quality 3D image data through a binocular stereo vision system and, combined with a refraction offset compensation algorithm, improve the accuracy and reliability of measurements in underwater environments. This method can directly capture images or videos of leopard-gill sea bass underwater and achieve accurate characterization measurements, possessing significant application value and the ability to significantly improve the level of intelligent aquaculture management.

[0138] It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. In addition, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0139] It will be readily understood by those skilled in the art that the above-described embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for characterizing and measuring the underwater leopard-gill spiny perch, characterized in that, The method includes: Underwater images of the leopard gill sea bass were acquired using a binocular camera, and key points of the leopard gill sea bass were detected using a key point detection model to obtain the pixel position of each key point in the underwater image. Establish an underwater light refraction model, including: In the formula, The pixel location of the key point in the underwater image. The key points correspond to the equivalent pixel positions in the air medium; The thickness of the waterproof cover for the binocular camera. This refers to the distance between the waterproof cover and the binocular camera. Let be the refractive index of water. The refractive index of the waterproof cover, The refractive index of air; Angle of incidence; By using an underwater light refraction model, the pixel positions of key points in the underwater image are refraction offset compensation, converting the pixel positions of key points in the underwater image into equivalent pixel positions in the air medium. By combining the equivalent pixel positions of key points in the air medium and the internal and external parameters of the binocular camera, the spatial position of each key point is obtained through stereo vision technology. Based on the spatial location data of each key point, characterization data of the underwater leopard gill spiny perch were determined.

2. The underwater leopard-gill spiny perch characterization and measurement method according to claim 1, characterized in that, The method also includes: Underwater images of the leopard gill perch were acquired using a binocular camera. First, the leopard gill perch were identified and its region was detected using a target detection model. Then, key points of the leopard gill perch were detected using a key point detection model, and the pixel positions of each key point in the underwater image were obtained.

3. The underwater leopard-gill spiny perch characterization and measurement method according to claim 1, characterized in that, Key features of the leopard gill spiny perch include: snout, eyes, upper caudal fin, lower caudal fin, anterior dorsal fin, posterior dorsal fin, upper dorsal fin, pelvic fin, and anal fin.

4. The underwater leopard-gill spiny perch characterization and measurement method according to claim 1, characterized in that, By combining the equivalent pixel positions of key points in the air medium and the intrinsic and extrinsic parameters of the binocular camera, the spatial positions of each key point are obtained through stereo vision technology, including: Establish a mapping relationship between spatial points and image points. Let any point B in space have projection points on the left and right camera image planes respectively. and The corresponding pixel coordinates are respectively and Then its homogeneous coordinates and the coordinates of point B in space... The relationship between them is described by the following formula: Among them, matrix elements The parameters represent the projection matrix of the camera, where k=1 and 2 represent the left and right cameras, respectively; and These represent the depth parameters of the left and right cameras, respectively; Transforming the above equations algebraically, we obtain a set of linear equations for X, Y, and Z: The coordinates of point B in space can be obtained by solving these two systems of linear equations. Thus, the spatial position of each key point is obtained by using the equivalent pixel position of the key point in the air medium and the internal and external parameters of the binocular camera.

5. The underwater leopard-gill spiny perch characterization and measurement method according to claim 1, characterized in that, Characteristic data for the underwater leopard gill spiny perch include body length, weight, and gill spine length.

6. An underwater characterization and measurement device for leopard-gill spiny perch, characterized in that, The device includes: The underwater pixel location unit is used to acquire underwater images of the leopard gill sea bass through a binocular camera, and to detect key points of the leopard gill sea bass through a key point detection model to obtain the pixel location of each key point in the underwater image. The light refraction model unit is used to build an underwater light refraction model, including: In the formula, The pixel location of the key point in the underwater image. The key points correspond to the equivalent pixel positions in the air medium; The thickness of the waterproof cover for the binocular camera. The distance between the waterproof cover and the binocular camera; Let be the refractive index of water. The refractive index of the waterproof cover, The refractive index of air; Angle of incidence; The refraction offset compensation unit is used to perform refraction offset compensation on the pixel position of key points in the underwater image through the underwater light refraction model, and convert the pixel position of key points in the underwater image into the equivalent pixel position in the air medium. The spatial position acquisition unit is used to combine the equivalent pixel position of the key point in the air medium and the internal and external parameters of the binocular camera to acquire the spatial position of each key point through stereo vision technology. The characterization data determination unit is used to determine the characterization data of the underwater leopard gill spiny perch based on the spatial location data of each key point.

7. A computer device, characterized in that, include: The processor and memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, implement the steps of the underwater leopard gill spiny perch characterization measurement method according to any one of claims 1 to 5.

8. A readable storage medium, characterized in that, It stores a program or instructions that, when executed by a processor, implement the steps of the underwater leopard gill spiny perch characterization and measurement method according to any one of claims 1 to 5.

Images