Three-dimensional model construction image compensation method for livestock individual phenotype determination

Through multi-angle, multi-light source image acquisition and three-dimensional laser scanning data fusion, combined with image compensation algorithm, the three-dimensional model is constructed and optimized, and the accuracy and efficiency problems of traditional methods are solved, and accurate determination and efficient processing of livestock morphology are achieved.

CN119006762AInactive Publication Date: 2024-11-22INSTITUTE OF ANIMAL SCIENCES OF CHINESE ACADEMY OF AGRICULTURAL SCIENCES
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202411171988.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional livestock phenotype measurement methods are affected by environmental factors, difficult to collect images, insufficient accuracy, low efficiency, and two-dimensional image analysis cannot fully reflect the three-dimensional morphological characteristics of livestock. The existing three-dimensional model construction methods are difficult to accurately match the actual morphology.

Method used

Images are collected by multi-angle and multi-light sources, combined with a three-dimensional laser scanner to acquire point cloud data, and through image stitching and data fusion, a three-dimensional model is built, and image compensation algorithm is used to optimize, and iteratively adjust until the needs are met.

Benefits of technology

It improves the accuracy and efficiency of livestock phenotype measurement, reduces the influence of environmental factors, and is suitable for livestock of various body types and types, with strong versatility and adaptability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006762A_ABST
    Figure CN119006762A_ABST
Patent Text Reader

Abstract

The invention discloses a three-dimensional model construction image compensation method for livestock individual phenotype determination, and the method comprises the steps: collecting and preprocessing a livestock image, and obtaining an initial livestock image; performing feature extraction on the initial livestock image to obtain morphological features of the livestock; constructing a three-dimensional model based on the morphological characteristics, and optimizing the three-dimensional model through an image compensation algorithm to obtain a target three-dimensional model; and performing comparative analysis on the target three-dimensional model and the actual livestock, verifying the accuracy of the model, and performing iterative optimization on the target three-dimensional model according to a verification result until requirements are met. The accuracy and efficiency of livestock individual phenotype determination can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of livestock individual phenotype measurement, and in particular relates to a three-dimensional model construction image compensation method for livestock individual phenotype measurement. Background Art

[0002] The determination of individual livestock phenotyping is an important basic work in animal husbandry, which involves livestock growth and development, genetic evaluation, disease diagnosis, etc. Traditional phenotyping methods mainly rely on manual measurement and image analysis, which have the following defects:

[0003] Influence of environmental factors: Environmental factors such as lighting and background will affect the image quality and thus affect the measurement results.

[0004] Difficulty in image acquisition: Livestock are large in size and move around in a wide range, making it difficult to capture comprehensive image coverage.

[0005] Lack of accuracy and low efficiency: Manual measurement is easily affected by subjective factors, resulting in inaccurate data. In addition, the manual measurement process is cumbersome, time-consuming, and inefficient, making it unsuitable for large-scale applications.

[0006] Limited data dimension: Two-dimensional image analysis cannot fully reflect the three-dimensional morphological characteristics of livestock, and the existing three-dimensional model construction methods are difficult to accurately match the actual morphology of livestock, resulting in deviations between the model and reality. Summary of the invention

[0007] In order to solve the above technical problems, the present invention provides a three-dimensional model construction image compensation method for livestock individual phenotype determination, comprising:

[0008] Collecting livestock images and performing preprocessing to obtain initial livestock images;

[0009] Extracting features from the initial livestock image to obtain morphological features of the livestock;

[0010] Building a three-dimensional model based on the morphological features, optimizing the three-dimensional model through an image compensation algorithm, and obtaining a target three-dimensional model;

[0011] The target three-dimensional model is compared and analyzed with actual livestock to verify the accuracy of the model, and the target three-dimensional model is iteratively optimized according to the verification result until the requirements are met.

[0012] Preferably, the process of collecting livestock images includes:

[0013] Use multi-angle, multi-light source depth cameras to obtain image data from different directions and under different lighting conditions at the same time and in different locations;

[0014] Scan the surface of livestock based on a 3D laser scanner to obtain point cloud data;

[0015] The image data and the point cloud data are registered and fused to obtain livestock images.

[0016] Preferably, the multi-angle, multi-light source depth camera uses a calibration plate to identify the coordinate system of the camera, and then automatically stitches the images using an image stitching algorithm;

[0017] The calibration plate is a cubic calibration plate, and the surface is frosted.

[0018] Preferably, the process of automatically stitching images using an image stitching algorithm includes: stitching images based on features;

[0019] The process of image stitching based on features includes:

[0020] The feature points are detected and extracted from the image by using the ORB algorithm, the descriptors are calculated for the area around each feature point, and the corresponding feature point pairs between the two images are obtained by using the FLANN matching algorithm;

[0021] The transformation relationship between images is calculated by matching feature point pairs to obtain the homography matrix;

[0022] The homography matrix is ​​further screened using the RANSAC algorithm to obtain a target homography matrix;

[0023] According to the target homography matrix, the image to be stitched is perspective transformed so that the image to be stitched is consistent with the reference image in perspective, and then the overlapped area of ​​the deformed images is processed by a Laplacian pyramid fusion method;

[0024] The maximum flow minimum cut method is used to automatically find the optimal stitching seam for image stitching to obtain the stitched image.

[0025] Preferably, the process of automatically stitching images using an image stitching algorithm further includes: stitching images based on regions;

[0026] The process of performing image stitching based on regions includes:

[0027] The reference image is divided into small blocks, and the sum of squares of light intensity differences is used as the matching criterion to find the best matching position for each small block in the image to be stitched;

[0028] The APAP algorithm is used to automatically find the optimal stitching seam for image stitching to obtain the stitched image.

[0029] Preferably, the process of performing data registration and data fusion on the image data and the point cloud data to obtain the livestock image comprises:

[0030] Filter and denoise the point cloud data, remove irrelevant background information and noise, and then extract edge features and corner features;

[0031] After distortion correction and color balance adjustment of the image data, the SURF algorithm is used to detect the corresponding feature points in the image;

[0032] Use the feature matching method of Kd tree search for initial registration to obtain the association information of corresponding feature points between point cloud data and image data;

[0033] According to the association information, an iterative closest point algorithm is used to perform fine registration, optimize the alignment between the point cloud data and the image data, and the root mean square error is used as an evaluation index to evaluate the registration quality;

[0034] The finely registered data are merged to create a fused dataset in a unified coordinate system.

[0035] Preferably, the process of extracting features from the initial livestock image to obtain morphological features of the livestock includes:

[0036] The edge and corner detection is performed through the Sobel operator and Harris corner detection to extract the edge and corner information of the image; then the texture and shape analysis is performed to extract the texture and shape features of the image;

[0037] Based on the edge and corner information of the image and the texture and shape features of the image, a convolutional neural network is used to automatically learn the deep features of the image, and a deep feature extraction unit is applied to extract the deep features;

[0038] The multi-scale analysis unit extracts features at different scales and the SIFT algorithm is used to calculate the scale space extreme value of the key points to extract the scale-invariant key points;

[0039] The depth feature is integrated with the key point to obtain a feature vector.

[0040] Preferably, the process of optimizing the three-dimensional model by using an image compensation algorithm includes:

[0041] The input two-dimensional image is subjected to noise removal and smoothing, and the depth information is extracted from the two-dimensional images of different viewing angles using a multi-view diffusion model;

[0042] The stereo matching algorithm is used to calculate the disparity of pixels between adjacent perspectives to obtain a depth map;

[0043] Convert the depth map into point cloud data, and filter the generated point cloud data to remove outliers and noise;

[0044] The processed point cloud data is converted into a 3D mesh model using the ISOMER algorithm. During the reconstruction process, color and geometric prior information are combined to ensure that the model is visually consistent with the input image.

[0045] According to the texture information of the original two-dimensional image, the three-dimensional model is texture mapped, and then the image compensation algorithm of the convolutional neural network is used to enhance the input image and complete the color of the invisible area.

[0046] Preferably, the process of removing noise from the input two-dimensional image includes:

[0047] By calculating the distance between each point in the point cloud and its surrounding neighboring points, points whose distance is less than a preset threshold are removed.

[0048] Compared with the prior art, the present invention has the following advantages and technical effects:

[0049] The method of the present invention can effectively improve the accuracy and efficiency of individual phenotyping of livestock. Specifically,

[0050] Improved accuracy: Through multi-angle, multi-light source image acquisition and image compensation algorithms, the impact of environmental factors on the measurement results is reduced, and the matching degree between the model and the actual morphology of livestock is improved.

[0051] Improved efficiency: Automated image acquisition and processing procedures greatly reduce manual operations and improve measurement efficiency.

[0052] Expanded scope of application: This method is applicable to livestock of various sizes and types and has strong versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0054] Figure 1 The figure is a schematic diagram of a method flow of an embodiment of the present invention. DETAILED DESCRIPTION

[0055] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0056] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0057] like Figure 1 As shown, this embodiment provides a three-dimensional model construction image compensation method for livestock individual phenotype determination, which is specifically applied to pig breeding farms. By real-time monitoring of individual phenotypes of pigs and taking into account special environments, such as: temperature, humidity, and light of the air in the pig house, it provides farmers with data support on the growth, health, and genetic characteristics of pigs. The following steps need to be included:

[0058] Collecting livestock images and performing preprocessing to obtain initial livestock images;

[0059] Extract features from the initial livestock images to obtain morphological features of the livestock;

[0060] Construct a three-dimensional model based on morphological features, optimize the three-dimensional model through image compensation algorithm, and obtain the target three-dimensional model;

[0061] The target 3D model is compared and analyzed with the actual livestock to verify the accuracy of the model, and the target 3D model is iteratively optimized based on the verification results until the requirements are met.

[0062] Furthermore, the process of collecting livestock images includes:

[0063] Use multi-angle, multi-light source depth cameras to obtain image data from different directions and under different lighting conditions at the same time and in different locations;

[0064] Scan the surface of livestock based on a 3D laser scanner to obtain point cloud data;

[0065] The image data and point cloud data are registered and fused to obtain livestock images.

[0066] Furthermore, a multi-angle and multi-light source depth camera uses a calibration plate to identify the camera's coordinate system, and then automatically stitches images using an image stitching algorithm.

[0067] The calibration plate is a cube calibration plate with a frosted surface.

[0068] Furthermore, the process of automatically stitching images by using an image stitching algorithm includes: stitching images based on features;

[0069] The process of feature-based image stitching includes:

[0070] The feature points are detected and extracted from the image by using the ORB algorithm, the descriptors are calculated for the area around each feature point, and the corresponding feature point pairs between the two images are obtained by using the FLANN matching algorithm;

[0071] The transformation relationship between images is calculated by matching feature point pairs to obtain the homography matrix;

[0072] The homography matrix is ​​further screened using the RANSAC algorithm to obtain the target homography matrix;

[0073] According to the target homography matrix, the image to be stitched is perspective transformed to make the image to be stitched consistent with the reference image, and then the overlapped area of ​​the deformed image is processed by the Laplacian pyramid fusion method;

[0074] The maximum flow minimum cut method is used to automatically find the optimal stitching seam for image stitching to obtain the stitched image.

[0075] Furthermore, the process of automatically stitching images by using an image stitching algorithm also includes: stitching images based on regions;

[0076] The process of region-based image stitching includes:

[0077] The reference image is divided into small blocks, and the sum of squares of light intensity differences is used as the matching criterion to find the best matching position for each small block in the image to be stitched;

[0078] The APAP algorithm is used to automatically find the optimal stitching seam for image stitching to obtain the stitched image.

[0079] Furthermore, in order to meet the special environmental requirements of the breeding farm, such as temperature, humidity, and light, the present embodiment uses the Maier Microvision M4 depth camera. This camera has the characteristics of high precision, high stability, and resistance to ambient light interference, and is suitable for breeding environments. At the same time, the project also considers the difficulty of camera integration to ensure compatibility with existing systems and platforms.

[0080] The process of using a multi-angle, multi-light source depth camera to obtain image data from different directions and under different lighting conditions at the same time in different directions includes:

[0081] Equipment selection and procurement: Choose depth cameras and brackets suitable for the application scenario, considering factors such as performance, cost, and supplier support.

[0082] Install the bracket: Fix the bracket in a suitable position according to actual needs and usage scenarios to ensure that the bracket is stable and does not shake.

[0083] Mount the depth camera: Mount the depth camera to the bracket according to the manufacturer's instructions, making sure the camera's lens is pointed correctly and can cover the desired viewing area.

[0084] Install the driver: Depending on the camera model and operating system, download and install the corresponding driver so that the computer can recognize and operate the camera.

[0085] Set parameters: Set the camera's working mode, resolution, frame rate and other parameters in the software to adapt them to application requirements. Specifically, the parameters include:

[0086] Field of view: FOV defines the range of space that the camera can capture. Depending on the application requirements, a wide or narrow field of view may be required.

[0087] Working distance: The effective working distance of a camera is also an important factor, especially in applications that require operation at a specific distance.

[0088] Integration difficulty: Consider how to integrate the depth camera into an existing system or platform, including software compatibility, hardware interface, and installation method.

[0089] Calibration and calibration: Calibrate and calibrate the depth camera to improve the accuracy of depth measurement. This may include focus adjustment, stereo matching algorithm optimization, and synchronization of external sensors.

[0090] Furthermore, the depth camera of this embodiment includes depth cameras with various working principles, such as time of flight (TOF), structured light, stereo vision (binocular), etc.

[0091] ToF (Time of flight): It is a method of depth measurement with centimeter-level accuracy. The ToF ranging method belongs to the two-way ranging technology, which mainly uses the flight time of the signal between two asynchronous transceivers (or reflected surfaces) to measure the distance between nodes.

[0092] Binocular stereo vision: It is an important form of machine vision. It is based on the principle of parallax and uses imaging equipment to obtain two images of the object under test from different positions. It calculates the position deviation between corresponding points in the images to obtain the three-dimensional geometric information of the object.

[0093] Structured light: There are two types: time structured light and space structured light. The common speckle structured light is the space coded structured light. To meet the needs of dynamic scenes, space coded structured light can be used.

[0094] In this embodiment, acquiring images through the depth camera includes a single camera triggering progress and multi-point camera triggering.

[0095] Among them, the triggering of a single camera is relatively simple and can be completed through the DEMO software provided by the manufacturer. Through the single camera manual trigger interface, manually trigger the photo to obtain point cloud data.

[0096] The multi-point camera is triggered by software, which controls the camera to shoot at the same time and obtain photos of livestock in different positions at the same time.

[0097] Furthermore, the image data and the point cloud data are registered and fused to obtain livestock images, including:

[0098] Filter and denoise the point cloud data, remove irrelevant background information and noise, and then extract edge features and corner features;

[0099] After distortion correction and color balance adjustment of the image data, the corresponding feature points are detected in the image using the SIFT algorithm or the SURF algorithm;

[0100] Use the feature matching method of Kd tree search for initial registration to obtain the association information of corresponding feature points between point cloud data and image data;

[0101] According to the associated information, the iterative closest point algorithm is used for fine registration to optimize the alignment between point cloud data and image data, and the root mean square error is used as an evaluation index to evaluate the registration quality;

[0102] The finely registered data are merged to create a fused dataset in a unified coordinate system.

[0103] Furthermore, the process of extracting features from the initial livestock image to obtain morphological features of the livestock includes:

[0104] The edge and corner detection units are used to detect edges and corners, and extract edge and corner information of the image. Specifically, the Canny edge detector is used to identify the edges in the image, the edge position is determined by calculating the magnitude and direction of the gradient, and the Harris corner detection is used to calculate the second-order moment of the image to identify the corners.

[0105] The texture analysis unit and the shape description unit are used to perform texture and shape analysis and extract the texture and shape features of the image.

[0106] Use convolutional neural networks to automatically learn the deep features of images, and apply deep feature extraction units to extract deeper features.

[0107] The multi-scale analysis unit extracts features at different scales to adapt to features of different sizes. Specifically, the SIFT algorithm is used to calculate the scale space extreme value of the key point to extract the scale-invariant key points.

[0108] All extracted features are integrated through the feature fusion unit to form a comprehensive feature vector.

[0109] The fused feature vector is output for subsequent 3D model construction and image compensation.

[0110] Furthermore, the process of optimizing the three-dimensional model by the image compensation algorithm includes:

[0111] The input two-dimensional image is subjected to noise removal and smoothing, and the depth information is extracted from the two-dimensional images of different viewing angles using a multi-view diffusion model;

[0112] The stereo matching algorithm is used to calculate the disparity of pixels between adjacent perspectives to obtain a depth map;

[0113] Convert the depth map into point cloud data, and filter the generated point cloud data to remove outliers and noise;

[0114] The processed point cloud data is converted into a 3D mesh model using the ISOMER algorithm. During the reconstruction process, color and geometric prior information are combined to ensure that the model is visually consistent with the input image.

[0115] According to the texture information of the original two-dimensional image, the three-dimensional model is texture mapped, and then the image compensation algorithm of the convolutional neural network is used to enhance the input image and complete the color of the invisible area.

[0116] Furthermore, the process of removing noise from the input two-dimensional image includes:

[0117] By calculating the distance between each point in the point cloud and its surrounding neighboring points, points whose distance is less than a preset threshold are removed. The basis of this method is that noise points are often isolated or low-density points that are far away from surrounding points.

[0118] The method of the present invention can effectively improve the accuracy and efficiency of individual phenotyping of livestock. Specifically,

[0119] Improved accuracy: Through multi-angle, multi-light source image acquisition and image compensation algorithms, the impact of environmental factors on the measurement results is reduced, and the matching degree between the model and the actual morphology of livestock is improved.

[0120] Improved efficiency: Automated image acquisition and processing procedures greatly reduce manual operations and improve measurement efficiency.

[0121] Expanded scope of application: This method is applicable to livestock of various sizes and types and has strong versatility and adaptability.

[0122] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A three-dimensional model construction image compensation method for livestock individual phenotype determination, characterized in that: include: Collecting livestock images and preprocessing them to obtain initial livestock images; Extracting features from the initial livestock image to obtain morphological features of the livestock; Building a three-dimensional model based on the morphological features, optimizing the three-dimensional model through an image compensation algorithm, and obtaining a target three-dimensional model; The target three-dimensional model is compared and analyzed with actual livestock to verify the accuracy of the model, and the target three-dimensional model is iteratively optimized according to the verification result until the requirements are met.

2. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 1, characterized in that: The process of collecting livestock images includes: Use multi-angle, multi-light source depth cameras to obtain image data from different directions and under different lighting conditions at the same time and in different locations; Scan the surface of livestock based on a 3D laser scanner to obtain point cloud data; The image data and the point cloud data are registered and fused to obtain livestock images.

3. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 2, characterized in that: The multi-angle, multi-light source depth camera uses a calibration plate to identify the coordinate system of the camera, and then automatically stitches the images using an image stitching algorithm; The calibration plate is a cubic calibration plate, and the surface is frosted.

4. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 3, characterized in that: The process of automatically stitching images by using an image stitching algorithm includes: stitching images based on features; The process of image stitching based on features includes: The feature points are detected and extracted from the image by using the ORB algorithm, the descriptors are calculated for the area around each feature point, and the corresponding feature point pairs between the two images are obtained by using the FLANN matching algorithm; The transformation relationship between images is calculated by matching feature point pairs to obtain the homography matrix; The homography matrix is ​​further screened using the RANSAC algorithm to obtain a target homography matrix; According to the target homography matrix, the image to be stitched is perspective transformed so that the image to be stitched is consistent with the reference image in perspective, and then the overlapped area of ​​the deformed images is processed by a Laplacian pyramid fusion method; The maximum flow minimum cut method is used to automatically find the optimal stitching seam for image stitching to obtain the stitched image.

5. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 3, characterized in that: The process of automatically stitching images using an image stitching algorithm also includes: stitching images based on regions; The process of performing image stitching based on regions includes: The reference image is divided into small blocks, and the sum of squares of light intensity differences is used as the matching criterion to find the best matching position for each small block in the image to be stitched; The APAP algorithm is used to automatically find the optimal stitching seam for image stitching to obtain the stitched image.

6. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 2, characterized in that: The process of performing data registration and data fusion on the image data and the point cloud data to obtain the livestock image includes: Filter and denoise the point cloud data, remove irrelevant background information and noise, and then extract edge features and corner features; After distortion correction and color balance adjustment of the image data, the SURF algorithm is used to detect the corresponding feature points in the image; Use the feature matching method of Kd tree search for initial registration to obtain the association information of corresponding feature points between point cloud data and image data; According to the association information, an iterative closest point algorithm is used to perform fine registration, optimize the alignment between the point cloud data and the image data, and the root mean square error is used as an evaluation index to evaluate the registration quality; The finely registered data are merged to create a fused dataset in a unified coordinate system.

7. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 1, characterized in that: The process of extracting features from the initial livestock image to obtain morphological features of the livestock includes: The edge and corner detection is performed through the Sobel operator and Harris corner detection to extract the edge and corner information of the image; then the texture and shape analysis is performed to extract the texture and shape features of the image; Based on the edge and corner information of the image and the texture and shape features of the image, a convolutional neural network is used to automatically learn the deep features of the image, and a deep feature extraction unit is applied to extract the deep features; The multi-scale analysis unit extracts features at different scales and the SIFT algorithm is used to calculate the scale space extreme value of the key points to extract the scale-invariant key points; The depth feature is integrated with the key point to obtain a feature vector.

8. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 1, characterized in that: The process of optimizing the three-dimensional model by using an image compensation algorithm includes: The input two-dimensional image is subjected to noise removal and smoothing, and the depth information is extracted from the two-dimensional images of different viewing angles using a multi-view diffusion model; The stereo matching algorithm is used to calculate the disparity of pixels between adjacent perspectives to obtain a depth map; Convert the depth map into point cloud data, and filter the generated point cloud data to remove outliers and noise; The processed point cloud data is converted into a 3D mesh model using the ISOMER algorithm. During the reconstruction process, color and geometric prior information are combined to ensure that the model is visually consistent with the input image. According to the texture information of the original two-dimensional image, the three-dimensional model is texture mapped, and then the image compensation algorithm of the convolutional neural network is used to enhance the input image and complete the color of the invisible area.

9. The three-dimensional model construction image compensation method for livestock individual phenotype determination according to claim 8, characterized in that: The process of removing noise from the input two-dimensional image includes: By calculating the distance between each point in the point cloud and its surrounding neighboring points, points whose distance is less than a preset threshold are removed.

Citation Information

Cited By

  • Traveling livestock individual identification method, device and equipment, storage medium and product

    CN119323817A

  • Texture generation method and system of three-dimensional character model and storage medium

    CN119359882A

  • Texture generation method, system and storage medium for three-dimensional character model

    CN119359882B

  • Method and system for improving incubation management efficiency of egg-carrying shrimps

    CN119648457A

  • Chicken head feature determination method based on 360-degree panoramic scanning

    CN119992589A