Traveling livestock individual identification method, device and equipment, storage medium and product

Abstract

The invention provides a traveling livestock individual identification method, device and equipment, a storage medium and a product, and relates to the technical field of image processing, and the method comprises the steps: obtaining a color image and a depth image of livestock passing through a livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; obtaining a target face three-dimensional point cloud of the livestock based on the depth image and the color image of the livestock; based on the target face three-dimensional point cloud of the livestock, acquiring a facial organ distance of the livestock; based on the color image of the livestock, acquiring face color distribution and face hair texture of the livestock; and performing individual identification on the livestock based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock. According to the invention, efficient, accurate and automatic advancing livestock individual identification without physical marking is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment, storage medium and product for individual recognition of moving livestock. Background Art

[0002] In current livestock breeding management, livestock individual identification mainly relies on traditional physical marking methods, such as ear tags, leg rings and tattoos, as well as manual experience for grouping. These methods are not only complex and inefficient, but also prone to identification errors or omissions. Physical tags also face the risk of being lost or damaged, which not only leads to failure of individual identification, but also makes it difficult to integrate with digital management systems, thereby increasing the difficulty of data recording and traceability, and limiting the improvement of management efficiency.

[0003] In order to overcome these problems, the relevant technology proposes to use RFID (Radio Frequency Identification) technology for livestock individual identification. RFID technology can automatically identify target objects through radio waves without direct contact or line of sight, thereby significantly improving the recognition efficiency and the ability of automated data collection.

[0004] However, RFID technology still faces some challenges in its application, such as high cost, limited recognition range, and easy damage to tags, which makes it difficult to fully adapt to the requirements of modern large-scale farms for efficient, accurate, and automated management. Summary of the invention

[0005] The present invention provides a method, device, equipment, storage medium and product for individually identifying livestock, so as to solve the technical defect that the individual identification technology of livestock in the prior art cannot fully meet the requirements of modern large-scale breeding farms for efficient, accurate and automated management.

[0006] The present invention provides a method for identifying individual moving livestock, the method comprising: Acquire color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; Based on the depth image and the color image of the livestock, obtaining a three-dimensional point cloud of a target face of the livestock; Based on the target facial three-dimensional point cloud of the livestock, obtaining the distance of the facial organs of the livestock; Based on the color image of the livestock, acquiring the facial color distribution and facial hair texture of the livestock; Based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock, individual livestock identification is performed.

[0007] According to a method for individually identifying livestock in motion provided by the present invention, the method of individually identifying livestock based on the facial organ distance, facial color distribution and facial hair texture of the livestock comprises: Generate a comprehensive facial feature vector of the livestock based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock; Determine the similarity between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vectors in the livestock individual identification database, wherein the livestock individual identification database includes livestock individual numbers of all known livestock and the known facial feature comprehensive vectors associated with the livestock individual numbers; Based on the similarity between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vector in the livestock individual identification database, the livestock is individually identified.

[0008] According to a method for individually identifying livestock provided by the present invention, the livestock is individually identified based on the similarity between the facial feature integrated vector of the livestock and the known facial feature integrated vector in the livestock individual identification database, including: Determining a minimum similarity among the similarities between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vectors in the livestock individual identification database; When the minimum similarity is not greater than a preset threshold, determining the livestock individual number of the livestock as the livestock individual number associated with the known facial feature comprehensive vector corresponding to the minimum similarity; When the minimum similarity is greater than a preset threshold, a target livestock individual number corresponding to the livestock is generated based on the livestock individual number generation rule, and the target livestock individual number corresponding to the livestock and the facial feature comprehensive vector are associated and stored in the livestock individual identification database.

[0009] According to a method for identifying individual walking livestock provided by the present invention, the facial organ distance includes the distance between eyes, the distance between nose and mouth, the distance between nose and ears, and the distance between nose and eyes; the facial color distribution includes a facial color histogram and a color mean of a facial key area; the facial key area includes an eye area, a nose area, and a mouth area; the facial hair texture includes a facial hair LBP histogram; The method of generating a comprehensive facial feature vector of the livestock based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock comprises: Obtaining a facial organ feature vector of the livestock based on the distance between the livestock's eyes, the distance between the nose and the mouth, the distance between the nose and the ears, and the distance between the nose and the eyes; Obtaining a facial color feature vector of the livestock based on the color mean of the eye region, the color mean of the nose region, the color mean of the mouth region and the facial color histogram; Based on the LBP histogram of the facial hair of the livestock, a feature vector of the facial hair of the livestock is obtained; Based on the contribution of the facial organ feature vector, the facial color feature vector and the facial hair feature vector of the livestock to the individual identification of the moving livestock, determine a first weight corresponding to the facial organ feature vector, a second weight corresponding to the facial color feature vector and a third weight corresponding to the facial hair feature vector; A comprehensive facial feature vector of the livestock is generated based on the first weight corresponding to the facial organ feature vector, the second weight corresponding to the facial color feature vector, and the third weight corresponding to the facial hair feature vector.

[0010] According to a method for identifying individual moving livestock provided by the present invention, the method of obtaining a target facial three-dimensional point cloud of the livestock based on the depth image and the color image of the livestock comprises: determining a livestock facial region in the color image of the livestock; Based on the registration relationship between the color image and the depth image, extracting the original facial three-dimensional point cloud corresponding to the livestock facial area from the depth image; Determine the centroid of the original three-dimensional facial point cloud of the livestock, and perform size standardization and point cloud alignment on the original three-dimensional facial point cloud of the livestock based on the centroid to obtain a pre-processed three-dimensional facial point cloud of the livestock; Based on the pre-processed facial three-dimensional point cloud of the livestock, a facial expression and action recognition model is applied to determine the facial expression and action of the livestock, wherein the facial expression and action recognition model is obtained by training an initial facial expression and action recognition model based on a first livestock facial three-dimensional point cloud corresponding to a standard facial expression and action and a second livestock facial three-dimensional point cloud corresponding to a non-standard facial expression and action, wherein the non-standard facial expression and action include a non-standard mouth expression and action, a non-standard eye expression and action, and a non-standard nose expression and action; In the case where the facial expression of the livestock is a non-standard facial expression, determining a point cloud region to be calibrated in the pre-processed three-dimensional facial point cloud of the livestock, and applying a point cloud deformation algorithm to calibrate the facial expression of the livestock in the point cloud region to obtain a target three-dimensional facial point cloud of the livestock; In the case where the facial expression of the livestock is a standard facial expression, the pre-processed facial three-dimensional point cloud of the livestock is used as the target facial three-dimensional point cloud of the livestock.

[0011] According to a method for identifying individual moving livestock provided by the present invention, the original facial three-dimensional point cloud of the livestock is subjected to size standardization and point cloud alignment based on the centroid to obtain a pre-processed facial three-dimensional point cloud of the livestock, including: Obtaining the Euclidean distance between each point in the original facial three-dimensional point cloud of the livestock and the centroid, and obtaining the maximum Euclidean distance among the Euclidean distances; Based on the centroid and the maximum Euclidean distance, scaling the original facial three-dimensional point cloud to obtain a size-standardized facial three-dimensional point cloud of the livestock; Performing data decentralization processing on the size-standardized facial three-dimensional point cloud of the livestock to obtain a data decentralized facial three-dimensional point cloud of the livestock; Determine a covariance matrix of the decentralized facial 3D point cloud of the livestock based on the distance between each point in the decentralized facial 3D point cloud of the livestock and the centroid; Performing eigenvalue decomposition on the covariance matrix to obtain the point cloud principal axis direction of the decentralized facial three-dimensional point cloud of the livestock; The point cloud principal axis direction of the decentralized facial three-dimensional point cloud of the livestock data is aligned with the standard coordinate system to obtain a pre-processed facial three-dimensional point cloud after point cloud alignment processing.

[0012] The present invention also provides a device for identifying individual traveling livestock, the device comprising: A first moving livestock individual recognition module is used to obtain color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; A second moving livestock individual recognition module is used to obtain a target facial three-dimensional point cloud of the livestock based on the depth image and the color image of the livestock; A third moving livestock individual recognition module is used to obtain the distance of the livestock's facial organs based on the target facial three-dimensional point cloud of the livestock; A fourth moving livestock individual recognition module is used to obtain the facial color distribution and facial hair texture of the livestock based on the color image of the livestock; The fifth moving livestock individual identification module is used to perform individual identification of livestock based on the facial organ distance, facial color distribution and facial hair texture of the livestock.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the above-mentioned methods for identifying individual traveling livestock is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for identifying individual traveling livestock as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for identifying individual traveling livestock.

[0016] The method, device, equipment, storage medium and product for individual identification of moving livestock provided by the present invention obtain color images and depth images of livestock through an RGB-D depth camera, and use the depth image and color image to generate a three-dimensional point cloud of the target face, so as to more accurately obtain the distance of the facial organs of the livestock, and then combine the facial color distribution and facial hair texture extracted from the color image to realize efficient, accurate and automatic individual identification of moving livestock without the need for physical markings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic flow chart of the method for identifying individual moving livestock provided by the present invention; Figure 2 It is a schematic diagram of the structure of the individual identification device for moving livestock provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Figure 1 FIG. 1 is a flow chart of a method for identifying individual moving livestock provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, obtaining color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel.

[0021] Here, the RGB-D depth camera installed at the livestock herding channel should ensure that when the livestock passes through the livestock herding channel, the RGB-D depth camera can capture the livestock's facial image at the best angle and distance. For example, the RGB-D depth camera is installed in the narrow part or turning of the livestock herding channel, and the monitoring area of ​​the RGB-D depth camera matches the livestock's walking path to ensure that the facial image of each livestock can be captured.

[0022] When livestock enters the monitoring area of ​​the RGB-D depth camera, the RGB-D depth camera can not only capture color images of the livestock (i.e., RGB images), but also measure the distance between the livestock and the RGB-D depth camera through specific sensors (such as infrared sensors or structured light sensors) to generate a depth image.

[0023] Step 120: Acquire a three-dimensional point cloud of a target face of the livestock based on the depth image and the color image of the livestock.

[0024] After obtaining the color image and depth image of the livestock, these two types of images are combined through image processing technology (such as image registration, depth map conversion, etc.) to generate a target facial 3D point cloud of a set of 3D coordinate points representing the facial shape of the livestock. The target facial 3D point cloud can accurately reflect the 3D structure of the livestock's face.

[0025] Step 130: acquiring the distance of the facial organs of the livestock based on the target facial three-dimensional point cloud of the livestock.

[0026] After obtaining the target facial 3D point cloud of the livestock, the facial feature point detection algorithm can be used to accurately locate the livestock's facial features, such as the center point of the eye, the center point of the nose, the boundary point of the mouth, and the specific position of the ear. After locating the livestock's facial features, the distance of the facial organs is calculated, such as the distance between the center points of the two eyes, the distance between the center point of the nose and the mouth, and the distance from the center point of the nose to the specific position of the ear.

[0027] Step 140, based on the color image of the livestock, obtain the facial color distribution and facial hair texture of the livestock.

[0028] Here, the facial color distribution of livestock reflects the color variations in different areas of the livestock's face, while the facial hair texture reflects the arrangement and thickness of the livestock's facial hair.

[0029] Specifically, the color values ​​of the three channels of red (R), green (G) and blue (B) for each pixel can be extracted from the color image of the livestock. Then the color mean of the key facial areas (such as the eye area, nose area and mouth area) is calculated. , its color mean can be calculated by the following formula: ;in, Key facial areas Medium Pixels The color vector [R,G,B].

[0030] The color distribution of the livestock face in the livestock color image is counted, and the color features are represented by an RGB histogram. The histogram is usually divided into multiple color intervals, each color interval represents the number of pixels within a certain color range. In this embodiment, the color histogram is generally expressed as: ;in, Indicates The number of pixels in a color interval.

[0031] Therefore, in this embodiment, the facial color distribution of the livestock may include the color means of multiple key areas of the face and an overall facial color histogram.

[0032] Furthermore, texture analysis can be performed on the color images of livestock to extract facial hair texture, such as the arrangement, thickness, direction and contrast of livestock hair in the color images.

[0033] In one example, the LBP (Local Binary Patterns) algorithm can be used. The LBP algorithm is an operator used to describe the local texture features of an image. The color image of livestock is processed by the LBP algorithm, and each hair pixel in the color image of the livestock is assigned an LBP value. These LBP values ​​reflect the facial hair texture of the livestock.

[0034] Step 150, performing individual identification of livestock based on the facial organ distance, facial color distribution and facial hair texture of the livestock.

[0035] In this embodiment, the facial organ distance, facial color distribution and facial hair texture of the livestock can be integrated into a comprehensive facial feature vector, and the livestock can be individually identified through the facial feature vector.

[0036] The method for identifying individual moving livestock proposed in this embodiment obtains color images and depth images of livestock through an RGB-D depth camera, and uses the depth image and color image to generate a three-dimensional point cloud of the target face, so as to more accurately obtain the distance of the facial organs of the livestock, and then combine the facial color distribution and facial hair texture extracted from the color image to achieve efficient, accurate and automatic individual moving livestock identification without the need for physical markings.

[0037] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0038] In some embodiments, the individual identification of livestock based on the facial organ distance, facial color distribution and facial hair texture of the livestock includes: Generate a comprehensive facial feature vector of the livestock based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock; Determine the similarity between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vectors in the livestock individual identification database, wherein the livestock individual identification database includes livestock individual numbers of all known livestock and the known facial feature comprehensive vectors associated with the livestock individual numbers; Based on the similarity between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vector in the livestock individual identification database, the livestock is individually identified.

[0039] In this embodiment, the facial organ distance, facial color distribution and facial hair texture of the livestock are combined to generate a complete facial feature comprehensive vector.

[0040] Here, the distance of the facial organs of the livestock includes the distance between the eyes, the distance between the nose and the mouth, the distance between the nose and the ears, and the distance between the nose and the eyes. The facial color distribution includes the facial color histogram and the color mean of the key facial areas, specifically, the color mean of the eye area, the color mean of the nose area, the color mean of the mouth area, and the facial color histogram. The facial hair texture is the facial hair LBP histogram obtained by statistically analyzing the LBP value of each facial hair pixel obtained by LBP algorithm processing, which will not be described in detail here.

[0041] In one example, the comprehensive facial feature vector of livestock can be obtained as follows: Obtaining a facial organ feature vector of the livestock based on the distance between the livestock's eyes, the distance between the nose and the mouth, the distance between the nose and the ears, and the distance between the nose and the eyes; Obtaining a facial color feature vector of the livestock based on the color mean of the eye region, the color mean of the nose region, the color mean of the mouth region and the facial color histogram; Based on the LBP histogram of the facial hair of the livestock, a feature vector of the facial hair of the livestock is obtained; Based on the contribution of the facial organ feature vector, the facial color feature vector and the facial hair feature vector of the livestock to the individual identification of the moving livestock, determine a first weight corresponding to the facial organ feature vector, a second weight corresponding to the facial color feature vector and a third weight corresponding to the facial hair feature vector; A comprehensive facial feature vector of the livestock is generated based on the first weight corresponding to the facial organ feature vector, the second weight corresponding to the facial color feature vector, and the third weight corresponding to the facial hair feature vector.

[0042] In this embodiment, a facial organ feature vector can be constructed by measuring the distance between the eyes, the distance between the nose and the mouth, the distance between the nose and the ears, and the distance between the nose and the eyes of the livestock. ,in, The distance between the eyes, is the distance between the nose and the mouth, is the distance between the nose and the ear, is the distance between the nose and the eyes.

[0043] The facial color feature vector can be obtained by calculating the color mean of the eye area, nose area and mouth area of ​​the livestock and calculating the color histogram of the entire face.

[0044] like ,in, is the mean color value of the eye area, is the mean color value of the nose area, is the mean color value of the mouth area, is the facial color histogram.

[0045] The local binary pattern (LBP) algorithm is used to extract the texture features of the pixels in the facial hair area to obtain the facial hair LBP histogram, and then the facial hair feature vector is constructed. .

[0046] Afterwards, based on the pre-evaluated facial organ feature vector , facial color feature vector and facial hair feature vector The contribution of each feature vector to the individual identification of the moving livestock is determined by weighting each feature vector. Finally, the facial organ feature vectors are combined according to the determined weights. , facial color feature vector and facial hair feature vector The weighted sum is performed to generate a comprehensive vector of livestock facial features. This comprehensive vector integrates multiple feature information of livestock faces and can more comprehensively reflect the facial features of livestock.

[0047] Here, a feature selection algorithm (such as recursive feature elimination, etc.) can be used to evaluate the contribution of each feature vector of each type of livestock to the final individual identification of walking livestock. Specifically, the contribution of each feature vector of each type of livestock is evaluated by removing the facial organ feature vector, facial color feature vector, and facial hair feature vector of each livestock sample in the livestock sample data set of each type of livestock one by one, and observing the degree of change in the individual identification accuracy of walking livestock in the final livestock sample data set.

[0048] In this embodiment, color images and depth images are collected for each livestock in the farm in advance, and the color images and depth images are processed by image processing technology to extract facial features of each livestock, such as facial organ distance, facial color distribution, and facial hair texture. Then, based on the extracted facial features, a known facial feature comprehensive vector of each livestock is constructed, and a mapping relationship between the livestock individual numbers of the known livestock and their corresponding known facial feature comprehensive vectors is established to obtain a pre-established livestock individual identification database.

[0049] The generated facial feature comprehensive vector is used to calculate the similarity with the known facial feature comprehensive vectors in the pre-established livestock individual identification database, such as Euclidean distance, cosine similarity, etc., to evaluate the similarity between the livestock facial feature comprehensive vector and each known facial feature comprehensive vector in the livestock individual identification database. Finally, the livestock is individually identified based on the calculated similarity.

[0050] In some embodiments, the individual identification of livestock based on the similarity between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vector in the livestock individual identification database includes: Determining a minimum similarity among the similarities between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vectors in the livestock individual identification database; When the minimum similarity is not greater than a preset threshold, determining the livestock individual number of the livestock as the livestock individual number associated with the known facial feature comprehensive vector corresponding to the minimum similarity; When the minimum similarity is greater than a preset threshold, a target livestock individual number corresponding to the livestock is generated based on the livestock individual number generation rule, and the target livestock individual number corresponding to the livestock and the facial feature comprehensive vector are associated and stored in the livestock individual identification database.

[0051] Here, the preset threshold is a pre-set similarity limit used to determine whether two facial feature comprehensive vectors can be considered as facial feature comprehensive vectors of the same individual.

[0052] Specifically, the minimum value of the similarities between the livestock's facial feature comprehensive vector and all known facial feature comprehensive vectors stored in the livestock individual identification database, ie, the minimum similarity, is determined.

[0053] If the minimum similarity is not greater than the preset threshold, it indicates that the livestock and the known livestock with the known facial feature comprehensive vector corresponding to the minimum similarity can be considered to belong to the same individual. Therefore, the individual number of the known livestock is assigned to the livestock.

[0054] If the minimum similarity is greater than a preset threshold, indicating that the livestock is not the same individual as the known livestock, the livestock individual number generation rule generates a new target livestock individual number for the livestock, and the newly generated target livestock individual number and its corresponding facial feature comprehensive vector are associated and stored in the livestock individual identification database. In this way, the livestock individual identification database contains the information of the newly identified livestock for future identification and management.

[0055] The method for individually identifying moving livestock proposed in this embodiment performs individual identification by calculating the similarity between the livestock's facial feature comprehensive vector and the known facial feature comprehensive vector in the livestock individual identification database, thereby achieving efficient, accurate and automated individual identification of moving livestock without the need for physical markings.

[0056] In some embodiments, the step of obtaining a target facial 3D point cloud of the livestock based on the depth image and the color image of the livestock includes: determining a livestock facial region in the color image of the livestock; Based on the registration relationship between the color image and the depth image, extracting the original facial three-dimensional point cloud corresponding to the livestock facial area from the depth image; Determine the centroid of the original three-dimensional facial point cloud of the livestock, and perform size standardization and point cloud alignment on the original three-dimensional facial point cloud of the livestock based on the centroid to obtain a pre-processed three-dimensional facial point cloud of the livestock; Based on the pre-processed facial three-dimensional point cloud of the livestock, a facial expression and action recognition model is applied to determine the facial expression and action of the livestock, wherein the facial expression and action recognition model is obtained by training an initial facial expression and action recognition model based on a first livestock facial three-dimensional point cloud corresponding to a standard facial expression and action and a second livestock facial three-dimensional point cloud corresponding to a non-standard facial expression and action, wherein the non-standard facial expression and action include a non-standard mouth expression and action, a non-standard eye expression and action, and a non-standard nose expression and action; In the case where the facial expression of the livestock is a non-standard facial expression, determining a point cloud region to be calibrated in the pre-processed three-dimensional facial point cloud of the livestock, and applying a point cloud deformation algorithm to calibrate the facial expression of the livestock in the point cloud region to obtain a target three-dimensional facial point cloud of the livestock; In the case where the facial expression of the livestock is a standard facial expression, the pre-processed facial three-dimensional point cloud of the livestock is used as the target facial three-dimensional point cloud of the livestock.

[0057] In this embodiment, a facial detection algorithm, such as a Haar feature classifier, is first used to identify the livestock facial region from the livestock color image. After the livestock facial region is identified, the depth value of the livestock facial region is extracted from the depth image using the registration relationship between the color image and the depth image, and the depth values ​​are converted into points in a three-dimensional space to form an original facial three-dimensional point cloud.

[0058] After obtaining the original facial 3D point cloud of the livestock, further processing is required to obtain the target facial 3D point cloud. First, it is necessary to determine the centroid of the original facial 3D point cloud, that is, the average position of all points in the original facial 3D point cloud. Then, the original facial 3D point cloud is size-standardized by the centroid to adjust the size of the point cloud to a fixed range. In addition to size standardization, the point cloud also needs to be aligned to arrange the points in the point cloud according to a certain rule to facilitate subsequent feature extraction and recognition. After size standardization and alignment, a preprocessed facial 3D point cloud with a fixed size and arrangement can be obtained.

[0059] In some embodiments, the step of performing size standardization and point cloud alignment on the original facial three-dimensional point cloud of the livestock based on the centroid to obtain the pre-processed facial three-dimensional point cloud of the livestock includes: Obtaining the Euclidean distance between each point in the original facial three-dimensional point cloud of the livestock and the centroid, and obtaining the maximum Euclidean distance among the Euclidean distances; Based on the centroid and the maximum Euclidean distance, scaling the original facial three-dimensional point cloud to obtain a size-standardized facial three-dimensional point cloud of the livestock; Performing data decentralization processing on the size-standardized facial three-dimensional point cloud of the livestock to obtain a data decentralized facial three-dimensional point cloud of the livestock; Determine a covariance matrix of the decentralized facial 3D point cloud of the livestock based on the distance between each point in the decentralized facial 3D point cloud of the livestock and the centroid; Performing eigenvalue decomposition on the covariance matrix to obtain the point cloud principal axis direction of the decentralized facial three-dimensional point cloud of the livestock; The point cloud principal axis direction of the decentralized facial three-dimensional point cloud of the livestock data is aligned with the standard coordinate system to obtain a pre-processed facial three-dimensional point cloud after point cloud alignment processing.

[0060] Specifically, the Euclidean distance between each point in the original 3D facial point cloud of the livestock and the centroid is first calculated. In this embodiment, the Euclidean distance between each point and the centroid is used to measure the spatial distribution of each point in the original 3D facial point cloud to the centroid. Then, the maximum Euclidean distance is found from these Euclidean distances, that is, the Euclidean distance from the point farthest from the centroid to the centroid in the original 3D facial point cloud. Finally, based on the centroid and the maximum Euclidean distance, the original 3D facial point cloud is scaled. Here, for each point in the original 3D facial point cloud, the position of each point is scaled to the unit sphere in the following manner to obtain a size-standardized 3D facial point cloud: ,in, is the scaled The location of the point, is the first The location of the point, is the location of the center of mass, is the maximum Euclidean distance.

[0061] After the size standardization is completed, the size-standardized facial 3D point cloud is decentralized to eliminate the absolute position information of the point cloud in space and only retain the relative position relationship within the point cloud. Then, based on the distance between each point and the centroid in the decentralized facial 3D point cloud of the livestock data, the covariance matrix of the decentralized facial 3D point cloud of the livestock data is determined. ,in, In this embodiment, the covariance matrix describes the spatial relationship between points in the point cloud, that is, the distribution and correlation in various directions.

[0062] Perform eigenvalue decomposition on the covariance matrix and decompose the covariance matrix into the form of eigenvalues ​​and eigenvectors. Through eigenvalue decomposition, the principal components (i.e., eigenvectors) of the point cloud in each direction and their corresponding importance (i.e., eigenvalues) can be obtained. Among them, the eigenvector with the largest eigenvalue corresponds to the direction of the point cloud principal axis, that is, the direction in which the point cloud extends the longest in space. After determining the point cloud principal axis direction of the data decentralized facial 3D point cloud, align the point cloud principal axis direction of the data decentralized facial 3D point cloud with the standard coordinate system. For example, align the first principal component (i.e., the eigenvector corresponding to the largest eigenvalue) to the x-axis, and align the second principal component to the y-axis, so that a preprocessed facial 3D point cloud with a fixed size, direction, and arrangement can be obtained.

[0063] Furthermore, after obtaining the pre-processed facial three-dimensional point cloud, the facial expression and action recognition model is combined to recognize the facial expression and action of livestock.

[0064] Here, the facial expression action recognition model is obtained by training the initial facial expression action recognition model using the first livestock facial 3D point cloud corresponding to the standard facial expression action and the second livestock facial 3D point cloud corresponding to the non-standard facial expression action. The initial facial expression action recognition model can use a deep learning architecture, such as a convolutional neural network or a graph neural network, to process the facial 3D point cloud. During the training process, the initial facial expression action recognition model learns to extract features from the input facial 3D point cloud and classify the facial expression action into standard or non-standard categories based on these features.

[0065] Specifically, the first animal face 3D point cloud corresponding to the standard facial expression action represents the face shape of the animal under the standard (or neutral) facial expression. The second animal face 3D point cloud corresponding to the non-standard facial expression action includes the face 3D point clouds under various non-standard facial expression actions, including but not limited to non-standard mouth expression actions (such as opening the mouth, pouting the lips, etc.), non-standard eye expression actions (such as closing the eyes, squinting, etc.), and non-standard nose expression actions (such as wrinkling the nose, raising the nose, etc.).

[0066] In this embodiment, the pre-processed facial 3D point cloud of the livestock is input into a trained facial expression and action recognition model, and the facial expression and action recognition model processes the pre-processed facial 3D point cloud and outputs the recognized facial expression and action type.

[0067] If the recognized facial expression is a non-standard facial expression, a region to be calibrated in the pre-processed facial 3D point cloud is determined. Here, the region to be calibrated is usually associated with facial feature points associated with the non-standard action, such as a point cloud of a mouth, eye or nose region.

[0068] Then, the point cloud deformation algorithm is applied to correct the area to be calibrated. Specifically, the point cloud deformation algorithm can deform the point cloud of non-standard actions in the area to be calibrated into a shape close to the standard based on the point cloud data of standard facial expression actions. During the deformation process, the topological structure and overall consistency of the point cloud are maintained to avoid unnatural shapes or distortions. The calibrated facial 3D point cloud is used as the target facial 3D point cloud of the livestock.

[0069] If the recognized facial expression movement is a standard facial expression movement, the pre-processed facial 3D point cloud is directly used as the target facial 3D point cloud of the livestock without correction.

[0070] The method for individual identification of moving livestock proposed in this embodiment converts the original facial 3D point cloud of the livestock into a pre-processed facial 3D point cloud with standard size, direction and arrangement through the above processing method. Furthermore, the pre-processed facial 3D point cloud of the livestock is subjected to facial expression and action recognition and calibration in combination with a facial expression and action recognition model to obtain the target facial 3D point cloud of the livestock, which is convenient for subsequent feature extraction and individual identification, thereby ensuring the accuracy and consistency of facial data of different livestock.

[0071] Based on any of the above embodiments, the present invention further provides a device for identifying individual moving livestock. Figure 2 Schematic diagram of the structure of the individual identification device for moving livestock provided by the present invention. Figure 2 As shown, the device comprises: A first moving livestock individual identification module 210 is used to obtain color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; A second moving livestock individual recognition module 220 is used to obtain a target facial three-dimensional point cloud of the livestock based on the depth image and the color image of the livestock; The third moving livestock individual recognition module 230 is used to obtain the distance of the livestock's facial organs based on the target facial three-dimensional point cloud of the livestock; The fourth moving livestock individual identification module 240 is used to obtain the facial color distribution and facial hair texture of the livestock based on the color image of the livestock; The fifth moving livestock individual recognition module 250 is used to perform individual recognition of livestock based on the facial organ distance, facial color distribution and facial hair texture of the livestock.

[0072] The device provided by the embodiment of the present invention obtains color images and depth images of livestock through an RGB-D depth camera, and uses the depth image and the color image to generate a three-dimensional point cloud of the target face, so as to more accurately obtain the distance of the facial organs of the livestock, and then combine the facial color distribution and facial hair texture extracted from the color image to realize efficient, accurate and automatic individual recognition of moving livestock without the need for physical markings.

[0073] The individual identification device for moving livestock described in this embodiment and the individual identification method for moving livestock provided by the present invention described above can be referred to each other, and will not be described in detail herein.

[0074] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communications interface 320 and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the individual identification method of traveling livestock, which includes: Acquire color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; Based on the depth image and the color image of the livestock, obtaining a three-dimensional point cloud of a target face of the livestock; Based on the target facial three-dimensional point cloud of the livestock, obtaining the distance of the facial organs of the livestock; Based on the color image of the livestock, acquiring facial color distribution and facial hair texture of the livestock; Based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock, individual livestock identification is performed.

[0075] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0076] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the individual identification method of traveling livestock provided by the above methods, the method includes: Acquire color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; Based on the depth image and the color image of the livestock, obtaining a three-dimensional point cloud of a target face of the livestock; Based on the target facial three-dimensional point cloud of the livestock, obtaining the distance of the facial organs of the livestock; Based on the color image of the livestock, acquiring facial color distribution and facial hair texture of the livestock; Based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock, individual livestock identification is performed.

[0077] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the method for identifying individual traveling livestock provided by the above methods, the method comprising: Acquire color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; Based on the depth image and the color image of the livestock, obtaining a three-dimensional point cloud of a target face of the livestock; Based on the target facial three-dimensional point cloud of the livestock, obtaining the distance of the facial organs of the livestock; Based on the color image of the livestock, acquiring facial color distribution and facial hair texture of the livestock; Based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock, individual livestock identification is performed.

[0078] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical coding feature diagrams therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying individual moving livestock, characterized in that: The method comprises: Acquire color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; Based on the depth image and the color image of the livestock, obtaining a three-dimensional point cloud of a target face of the livestock; Based on the target facial three-dimensional point cloud of the livestock, obtaining the distance of the facial organs of the livestock; Based on the color image of the livestock, acquiring the facial color distribution and facial hair texture of the livestock; Based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock, individual livestock identification is performed.

2. The method for identifying individual moving livestock according to claim 1, characterized in that: The method of individually identifying livestock based on the facial organ distance, facial color distribution and facial hair texture of the livestock includes: Generate a comprehensive facial feature vector of the livestock based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock; Determine the similarity between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vectors in the livestock individual identification database, wherein the livestock individual identification database includes livestock individual numbers of all known livestock and the known facial feature comprehensive vectors associated with the livestock individual numbers; Based on the similarity between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vector in the livestock individual identification database, the livestock is individually identified.

3. The method for identifying individual moving livestock according to claim 2, characterized in that: The method of individually identifying livestock based on the similarity between the livestock's facial feature integrated vector and known facial feature integrated vectors in a livestock individual identification database comprises: Determining a minimum similarity among the similarities between the facial feature comprehensive vector of the livestock and the known facial feature comprehensive vectors in the livestock individual identification database; When the minimum similarity is not greater than a preset threshold, determining the livestock individual number of the livestock as the livestock individual number associated with the known facial feature comprehensive vector corresponding to the minimum similarity; When the minimum similarity is greater than a preset threshold, a target livestock individual number corresponding to the livestock is generated based on the livestock individual number generation rule, and the target livestock individual number corresponding to the livestock and the facial feature comprehensive vector are associated and stored in the livestock individual identification database.

4. The method for identifying individual moving livestock according to claim 2, characterized in that: The facial organ distance includes the distance between eyes, the distance between nose and mouth, the distance between nose and ears, and the distance between nose and eyes; the facial color distribution includes a facial color histogram and a color mean of a facial key area; the facial key area includes an eye area, a nose area, and a mouth area; the facial hair texture includes a facial hair LBP histogram; The method of generating a comprehensive facial feature vector of the livestock based on the facial organ distance, the facial color distribution and the facial hair texture of the livestock comprises: Obtaining a facial organ feature vector of the livestock based on the distance between the livestock's eyes, the distance between the nose and the mouth, the distance between the nose and the ears, and the distance between the nose and the eyes; Obtaining a facial color feature vector of the livestock based on the color mean of the eye region, the color mean of the nose region, the color mean of the mouth region and the facial color histogram; Based on the LBP histogram of the facial hair of the livestock, a feature vector of the facial hair of the livestock is obtained; Based on the contribution of the facial organ feature vector, the facial color feature vector and the facial hair feature vector of the livestock to the individual identification of the moving livestock, determine a first weight corresponding to the facial organ feature vector, a second weight corresponding to the facial color feature vector and a third weight corresponding to the facial hair feature vector; A comprehensive facial feature vector of the livestock is generated based on the first weight corresponding to the facial organ feature vector, the second weight corresponding to the facial color feature vector, and the third weight corresponding to the facial hair feature vector.

5. The method for identifying individual traveling livestock according to claim 1, characterized in that: The step of obtaining a target facial three-dimensional point cloud of the livestock based on the depth image and the color image of the livestock comprises: determining a livestock facial region in the color image of the livestock; Based on the registration relationship between the color image and the depth image, extracting the original facial three-dimensional point cloud corresponding to the livestock facial area from the depth image; Determine the centroid of the original three-dimensional facial point cloud of the livestock, and perform size standardization and point cloud alignment on the original three-dimensional facial point cloud of the livestock based on the centroid to obtain a pre-processed three-dimensional facial point cloud of the livestock; Based on the pre-processed facial three-dimensional point cloud of the livestock, a facial expression and action recognition model is applied to determine the facial expression and action of the livestock, wherein the facial expression and action recognition model is obtained by training an initial facial expression and action recognition model based on a first livestock facial three-dimensional point cloud corresponding to a standard facial expression and action and a second livestock facial three-dimensional point cloud corresponding to a non-standard facial expression and action, wherein the non-standard facial expression and action include a non-standard mouth expression and action, a non-standard eye expression and action, and a non-standard nose expression and action; In the case where the facial expression of the livestock is a non-standard facial expression, determining a point cloud region to be calibrated in the pre-processed three-dimensional facial point cloud of the livestock, and applying a point cloud deformation algorithm to calibrate the facial expression of the livestock in the point cloud region to obtain a target three-dimensional facial point cloud of the livestock; In the case where the facial expression of the livestock is a standard facial expression, the pre-processed facial three-dimensional point cloud of the livestock is used as the target facial three-dimensional point cloud of the livestock.

6. The method for identifying individual moving livestock according to claim 5, characterized in that: The step of performing size standardization and point cloud alignment processing on the original facial three-dimensional point cloud of the livestock based on the centroid to obtain a pre-processed facial three-dimensional point cloud of the livestock includes: Obtaining the Euclidean distance between each point in the original facial three-dimensional point cloud of the livestock and the centroid, and obtaining the maximum Euclidean distance among the Euclidean distances; Based on the centroid and the maximum Euclidean distance, scaling the original facial three-dimensional point cloud to obtain a size-standardized facial three-dimensional point cloud of the livestock; Performing data decentralization processing on the size-standardized facial three-dimensional point cloud of the livestock to obtain a data decentralized facial three-dimensional point cloud of the livestock; Determine a covariance matrix of the decentralized facial 3D point cloud of the livestock based on the distance between each point in the decentralized facial 3D point cloud of the livestock and the centroid; Performing eigenvalue decomposition on the covariance matrix to obtain the point cloud principal axis direction of the decentralized facial three-dimensional point cloud of the livestock; The point cloud principal axis direction of the decentralized facial three-dimensional point cloud of the livestock data is aligned with the standard coordinate system to obtain a pre-processed facial three-dimensional point cloud after point cloud alignment processing.

7. A device for identifying individual moving livestock, characterized in that: The device comprises: A first moving livestock individual recognition module is used to obtain color images and depth images of livestock passing through the livestock grouping channel based on an RGB-D depth camera installed at the livestock grouping channel; A second moving livestock individual recognition module is used to obtain a target facial three-dimensional point cloud of the livestock based on the depth image and the color image of the livestock; A third moving livestock individual recognition module is used to obtain the distance of the livestock's facial organs based on the target facial three-dimensional point cloud of the livestock; A fourth moving livestock individual recognition module is used to obtain the facial color distribution and facial hair texture of the livestock based on the color image of the livestock; The fifth moving livestock individual identification module is used to perform individual identification of livestock based on the facial organ distance, facial color distribution and facial hair texture of the livestock.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for individually identifying traveling livestock according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for individually identifying traveling livestock as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for individually identifying traveling livestock as claimed in any one of claims 1 to 6 is implemented.

Images