An efficient multi-scale sheep face detection method

Abstract

The invention provides an efficient multi-scale sheep front face detection method, comprising: step S1, collecting data set images containing sheep front faces of different scales; step S2, marking sheep face frames and 7 facial feature points in the data set images of the sheep front faces, and obtaining a marked data set; step S3, randomly dividing the marked data set into a training set, a verification set and a test set; step S4, inputting the training set into an efficient face and feature point detection network for training, and obtaining a trained network model; step S5, pre-processing a picture to be detected and a video key frame, and then inputting the pre-processed picture and a video key frame into the trained network model, and obtaining a sheep face detection result; step S6, screening the detected sheep front face according to whether the 7 facial feature points of the sheep front face are detected and whether the detection frame and the 7 facial feature points meet the positive angle feature relationship.

Description

Technical Field

[0001] The invention belongs to the technical field of animal husbandry image processing, relates to sheep face recognition, and specifically relates to an efficient multi-scale sheep front face detection method. Background Art

[0002] Animal husbandry is a traditional industry in my country and a leading industry in agriculture and rural economy. Dairy sheep and mutton sheep farming are important components of the entire agriculture. As of 2020, the number of sheep in my country is 306.55 million. In the 1980s, my country introduced the concept of industrialized breeding from abroad and gradually realized the transformation of breeding production methods. In recent years, sheep breeding has gradually changed from small-scale family breeding to large-scale breeding in intensive sheep farms, realizing the scale of sheep breeding.

[0003] Although sheep farming has achieved scale, it has also brought some problems. At present, first of all, the epidemic prevention measures of some sheep farms are not in place, and the vaccination in the sheep farms is chaotic, which may lead to concentrated outbreaks of infectious diseases. Secondly, some sheep farms have simple and extensive feeding methods, unreasonable nutrition matching, and cannot achieve intensive feeding, resulting in high incidence rates. At the same time, the breeding management of some sheep farms is chaotic, there is inbreeding, and the quality of offspring is affected to a certain extent. In order to solve the above problems, the individual identification of sheep has changed from traditional identification methods represented by body color tags and branding to wearing electronic radio frequency ear tags. At present, most sheep farms manage sheep through ear tags. However, according to investigations, the use of ear tags still has many adverse effects on the production of sheep farms. For example, due to the lack of strict disinfection of ear tags or the tearing off of ear tags, it is easy to cause ear infection and rot in sheep, affecting the health of sheep; the large-scale use of ear tags will increase the breeding cost; the replacement of ear tags and the collection of ear tag information will consume a lot of manpower and material resources, reducing the economic benefits of sheep farms. Therefore, it is necessary to develop a new type of non-contact sheep identification method that can improve the economic benefits of sheep farms. At present, using deep learning technology to realize sheep face recognition is the main way to realize sheep individual recognition. However, due to the low accuracy and efficiency of the detection steps in the recognition process, the subsequent recognition links are directly inefficient. Therefore, the present invention aims to solve the problem of low efficiency and low accuracy in the sheep face detection link.

[0004] Sheep face detection is an important prerequisite for sheep face recognition. Sheep face recognition first extracts the sheep face area through the detection step, and then extracts and compares the features of the facial area to determine the individual sheep. Therefore, sheep face detection is a key step that affects the efficiency and accuracy of the recognition process. Experiments have found that most of the existing detection methods can detect the side face of sheep, resulting in different recognition results for the same sheep at the same time, and the need to add additional screening links, which greatly affects the recognition accuracy and overall recognition efficiency of sheep faces. At the same time, most sheep farms currently deploy sheep face recognition devices in narrow passages, resulting in a large number of sheep faces of different scales in the actual detection scene, and the existing methods are difficult to detect sheep faces with large scale differences in the picture, which also greatly affects the detection and recognition efficiency. Therefore, there are the following deficiencies in the existing research and technology:

[0005] (1) In the existing research, there is no dedicated detection algorithm for the seven key features of the sheep face in the sheep face detection step, which greatly affects the accuracy and efficiency of sheep face detection.

[0006] (2) There is a lack of methods to construct a multi-scale sheep face dataset under actual detection scenario conditions.

[0007] (3) The sheep face detection step of the existing method has the problem of low detection efficiency, which is difficult to meet the needs of real-time detection in large sheep farms. Some algorithms cannot achieve high detection accuracy while improving detection efficiency.

[0008] (4) The sheep face detection step of the existing method does not solve the problem of multi-scale sheep face detection in actual scenes. Summary of the invention

[0009] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an efficient multi-scale sheep face detection method to solve the technical problem that the efficiency and accuracy of the sheep face detection method in the prior art need to be further improved.

[0010] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:

[0011] An efficient multi-scale sheep face detection method is implemented in the following steps:

[0012] Step S1, collecting a dataset of images of sheep faces of different scales;

[0013] Step S2, marking the sheep face frame and seven facial feature points in the dataset image of the sheep's front face to obtain a marked dataset;

[0014] The seven facial feature points are the left eye corner point of the left eye, the right eye corner point of the left eye, the left eye corner point of the right eye, the right eye corner point of the right eye, the left nose wing, the right nose wing and the nose tip;

[0015] Step S3, randomly divide the labeled data set into a training set, a validation set, and a test set;

[0016] Step S4, inputting the training set into an efficient face and feature point detection network for training to obtain a trained network model;

[0017] The efficient face and feature point detection network is an improved RetinaFace algorithm;

[0018] Step S5, pre-processing the image and video key frame to be detected and inputting them into the trained network model to obtain the sheep face detection result;

[0019] The sheep face detection result includes 7 facial feature points and a detection frame of the sheep's front face;

[0020] Step S6, filtering the detected sheep's front face according to whether the seven facial feature points of the sheep's front face are detected and whether the detection frame and the seven facial feature points satisfy a positive angle feature relationship;

[0021] The formula for the positive angle characteristic relationship is as follows,

[0022]

[0023] Where:

[0024] Shape The symbol represents the feature point p m With p n distance;

[0025] Shape The symbol indicates the endpoint of the detection box D m With feature point p n distance;

[0026] α1 represents the positive angle facial proportion parameter;

[0027] α2 represents the adaptation parameter between the sheep’s facial frame and facial feature points;

[0028] α3 represents the facial symmetry balance parameter.

[0029] Compared with the prior art, the present invention has the following technical effects:

[0030] (I) The method of the present invention has high detection accuracy and short running time on a multi-scale test set simulating actual detection scenes. It is an efficient multi-scale sheep face detection method that is suitable for real-time multi-scale detection in actual sheep farm scenes.

[0031] (II) The multi-scale feature extraction capability implemented in the detection algorithm of the present invention can meet the needs of multi-scale detection of sheep in an actual sheep farm environment.

[0032] (III) The lightweight backbone network and the optimization of loss calculation adopted by the algorithm of the present invention ensure that the sheep face detection step remains highly efficient and can meet the needs of real-time detection.

[0033] (IV) The present invention can maintain a high accuracy rate of sheep face detection through further screening while meeting the requirements of efficient detection. The effective detection of sheep face provides accuracy guarantee for subsequent recognition steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the structure of the efficient face and feature point detection network of the present invention.

[0035] Figure 2 Detection results of sheep side face pictures by the method of comparative example 1, comparative example 2 and the embodiment of the present invention.

[0036] Figure 3 The following are the detection results of the front and side faces of sheep using the method of the present invention.

[0037] The specific contents of the present invention are further explained in detail below in conjunction with embodiments. DETAILED DESCRIPTION

[0038] It should be noted that, unless otherwise specified, all algorithms, modules, layers and functions in the present invention adopt algorithms, modules, layers and functions known in the prior art.

[0039] In accordance with the above technical scheme, specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical scheme of this application fall within the protection scope of the present invention.

[0040] Example:

[0041] This embodiment provides an efficient multi-scale sheep face detection method, which is performed according to the following steps:

[0042] Step S1, collecting a dataset of images containing sheep faces of different scales.

[0043] Step S1 specifically includes the following steps:

[0044] Step S101, use a camera to shoot a video of sheep in a farm. When shooting the video of the sheep, the front face of the sheep must be included. In the video, the sheep's face is located in different positions of the image, not a single middle position. The sheep's face is distributed in five positions of the upper left, lower left, upper right, lower right and middle of the original image; and the collected image contains sheep faces of multiple scales.

[0045] The multi-scale is divided into three types: large scale, medium scale and small scale; the large scale accounts for 1 / 4 of the image ratio; the medium scale accounts for 1 / 10 of the image ratio; and the small scale accounts for 1 / 20 of the image ratio.

[0046] Step S102, processing the video of the sheep, capturing key frames in the video, saving the image containing all the features of the complete front face of the sheep, and obtaining a data set image of the front face of the sheep.

[0047] All the features of the sheep's front face include the features of the left corner and the right corner of the left eye, the features of the left corner and the right corner of the right eye, the features of the left nose, the features of the right nose and the features of the nose tip.

[0048] In this embodiment, a video of sheep is processed, key frames in the video are captured, and images containing all the features of the sheep's front face are saved. The total sheep front face data set contains N pictures, where N is 3000. The pictures in the front face data set may contain multiple sheep faces, but must contain at least one sheep face that contains all the features of the sheep's front face.

[0049] Step S2, marking the sheep's facial frame and seven facial feature points in the dataset image of the sheep's front face to obtain a marked dataset.

[0050] Step S2 specifically includes the following steps:

[0051] Step S201 , annotating the dataset image of the sheep's front face, including annotating a square sheep face frame and seven facial feature points.

[0052] In the process of marking the sheep face frame, the upper left coordinate and the lower right coordinate of the frame are recorded.

[0053] The seven facial feature points are the left eye corner point of the left eye, the right eye corner point of the left eye, the left eye corner point of the right eye, the right eye corner point of the right eye, the left nose wing, the right nose wing and the nose tip.

[0054] Step S202, import the annotated coordinates into a text file, each picture record in the text file contains two lines of information; the first line starts with # and contains the absolute path of the picture; the second line contains the upper left coordinates, lower right coordinates, left eye corner coordinates, left eye right eye corner coordinates, right eye left eye corner coordinates, right eye right eye corner coordinates, left nose wing point coordinates, right nose wing point coordinates and nose tip point coordinates of the sheep face frame; the coordinates are stored according to x and y, and each coordinate point is separated by 1 or 0, 1 indicates that the point has high credibility and needs to be adopted, and 0 indicates that the credibility is low and should not be adopted, so as to obtain a labeled data set.

[0055] Step S3, randomly divide the labeled data set into a training set, a validation set, and a test set.

[0056] In this embodiment, the dataset is randomly divided into a training set, a validation set, and a test set in a ratio of 6:2:2. The training set contains 1,800 images, the validation set contains 600 images, and the test set contains 600 images; the training set, validation set, and test set described in step S3 all contain multi-scale sheep frontal face images.

[0057] Step S4, inputting the training set into an efficient face and feature point detection network for training to obtain a trained network model.

[0058] Specifically, in step S4, the efficient face and feature point detection network is an improved RetinaFace algorithm, and the specific structural diagram is as follows: Figure 1 As shown, Figure 1 In the figure, SAC represents the separable atrous convolution module; SE represents the squeeze-excitation module; Conv2d represents the two-dimensional convolution module; FPN represents the feature pyramid network; SSH represents the single-stage headless detector module; and CA represents the channel enhancement module.

[0059] Specifically, in step S4, the improved method of the RetinaFace algorithm includes the following steps:

[0060] Step S401, replacing the backbone network of the original RetinaFace algorithm with an improved lightweight backbone network MobileNetV3 large, the improvement process of the lightweight backbone network MobileNetV3 large is:

[0061] The layer-by-layer convolutions in all 3×3 depth-wise separable convolutions in the lightweight backbone network MobileNetV3 large are replaced with switchable dilated convolutions. The dilation rate of the dilated convolutions is adjusted through the switch operation during training to achieve soft switching of the convolution calculations. This allows the lightweight backbone network MobileNetV3 large to obtain receptive fields of different scales, which is beneficial for multi-scale detection.

[0062] The Switch operation is composed of a 5*5 average pooling layer and a 1*1 ordinary convolution layer;

[0063] In this embodiment, the structure of the switchable atrous convolution increases the receptive field of the network during the training process, so that the network can adapt to detecting targets of different scales and accurately locate the targets.

[0064] Step S402, adjusting the attention module configuration scheme in the network architecture of the lightweight backbone network MobileNetV3 large;

[0065] The adjustment method is as follows: adding an attention mechanism module to the 1st to 6th layers and the 11th to 15th layers of the bneck module in the lightweight backbone network MobileNetV3 large;

[0066] Specifically, the attention mechanism module is composed of a global pooling layer, a fully connected layer, a ReLU function, a fully connected layer, and a Sigmoid function;

[0067] The above adjustments in this embodiment enable the lightweight backbone network MobileNetV3 large to adapt to feature extraction of multi-scale objects, so that the lightweight backbone network MobileNetV3 large can automatically obtain the importance of each feature channel through learning, and then increase the weight of useful facial features according to the learning results, and suppress the weight of other feature layers;

[0068] Step S403, adding a channel strengthening module to the three-layer context modules of the SSH structure in the original RetinaFace algorithm;

[0069] The channel enhancement module consists of two parts, the first part consists of a horizontal average pooling layer, a 1*1 ordinary convolution layer, a 1*1 ordinary convolution layer, a normalization layer, a nonlinear layer, a 1*1 ordinary convolution layer and a Sigmoid function; the second part consists of a vertical average pooling layer, a 1*1 ordinary convolution layer shared with the first part, a normalization layer and a nonlinear layer shared with the first part, a 1*1 ordinary convolution layer, a 1*1 ordinary convolution layer and a Sigmoid function;

[0070] The channel enhancement module added in this embodiment is used to improve the ability to extract important features after fusing context information.

[0071] Step S404, optimize the loss function in the original RetinaFace algorithm, and the final loss function is shown as follows:

[0072]

[0073] Where:

[0074] L is the final loss function;

[0075] i is the anchor box;

[0076] is the classification loss function, which is composed of the softmax binary classification loss function;

[0077] c i is the probability that the predicted anchor box i contains a sheep face;

[0078] is the probability that the actual anchor box i contains a sheep face, and its value is 1 or 0. A value of 1 means that the actual box contains a sheep face, and a value of 0 means that the actual box does not contain a sheep face;

[0079] is the detection box position loss function, which is composed of the smooth-L1 function;

[0080] b i is the prediction parameter of the detection box position loss function, including the predicted center coordinate value b of the anchor box i x 、b y , and the width and height of anchor box i b w 、b h ;

[0081] is the actual parameter of the detection box position loss function, including the center coordinate value of the actual annotation box i And width and height

[0082] The loss function for detecting facial landmarks is composed of the smooth-L1 function;

[0083] l i To detect the predicted parameters of the facial feature point loss function, including the predicted coordinate values ​​of the seven facial feature point coordinates

[0084] To detect the actual parameters of the facial feature point loss function, including the coordinate values ​​of the seven facial feature points actually marked

[0085] λ1 is 0.5;

[0086] λ2 is 0.2.

[0087] Specifically, in step S4, the accuracy and recall rate of the training model are tested through the validation set every 5 rounds of training to verify the model detection effect; the final model detection accuracy of the trained model is tested through the test set, and the model is considered acceptable if and only if the model accuracy is higher than 95%.

[0088] Step S5, pre-processing the image to be detected and the key frame of the video and inputting them into the trained network model to obtain the sheep face detection result.

[0089] The sheep face detection result includes 7 facial feature points and a detection frame of the sheep's front face;

[0090] Specifically, in step S5, the preprocessing is: scaling the width of the image to 320 pixels and compressing the height of the image proportionally according to the original image ratio.

[0091] Specifically, step S5 includes the following steps:

[0092] In step S501, the detection image is preprocessed to scale the width of the image to 320 pixels and compress the height of the image proportionally according to the original image ratio before inputting the RGB three-channel image into the improved backbone network.

[0093] In step S502, the input RGB three-channel image is input into the backbone network and passes through the first 16 switchable hole convolutions of the backbone network; the 2nd to 16th convolutions adopt the bneck structure, and the SE channel attention module is used in the 2nd, 3rd, 4th, 5th, 6th, 7th, 12th, 13th, 14th, 15th, and 16th convolutions.

[0094] Step S503, the input image enters the improved network and the backbone network obtains three output feature layers and enters the FPN structure. The first feature layer uses 8 convolutions with kernel sizes of 1*1, 3*3, 3*3, 3*3, 5*5, 5*5, 5*5, and 3*3 to obtain a feature map of size 80*80*80; the second feature layer uses 6 convolutions with kernel sizes of 3*3, 3*3, 3*3, 3*3, 3*3, 5*5 to obtain a feature map of size 160*40*40; the third feature layer uses 3 convolutions with kernel sizes of 5*5, 5*5, and 3*3 to obtain a feature map of size 960*20*20. The output feature maps of the three feature layers are processed by the average pooling layer and the hardswitch activation function to obtain a dictionary containing three feature maps.

[0095] Step S504, input the output layers of the backbone network S3, S4, and S5 modules into FPNS1, FPN S2, and FPN S3. The specific steps are as follows: the output layer of the S5 module first undergoes 1*1 convolution to change the number of channels of the feature layer, and then the obtained feature layer undergoes 3*3 convolution and is input into the SSH module. The output layer of the S5 module that has undergone 1*1 convolution is upsampled (the upsampling uses the nearest neighbor interpolation method), and is added to the corresponding position of the output layer of the S4 module that has undergone 1*1 convolution. The obtained feature layer is input into the SSH module after 3*3 convolution. The above-mentioned FPN S2 output result is upsampled and added to the corresponding position of the output layer of the S3 module that has undergone 1*1 convolution. The obtained feature layer is input into the SSH module after 3*3 convolution. After the above three steps, three feature layers of different scales are output.

[0096] Step S505, the feature layers representing features of three different scales output by the FPN feature pyramid are input into the context module in the SSH module to which the channel enhancement module is added to obtain three feature layers having the same number of channels of 128 for representing features of different scales.

[0097] Step S506, input the three feature layers obtained in step 5 into the detector respectively. The detector includes three detectors: ClassHead, BboxHead and LandmarkHead. In the ClassHead module, a 1*1 convolution is first performed, and then the LogSoftmax function is performed to obtain the ClassHead module output. In the BboxHead module, a 1*1 convolution is performed to obtain the BboxHead module output. In the LandmarkHead module, a 1*1 convolution is performed to obtain the LandmarkHead module output.

[0098] Step S507, calculate the score according to the output matrix obtained by the above ClassHead, BboxHead and LandmarkHead, compare the set score threshold to determine whether the detection box is included, the detection box position and the positions of the seven facial feature points, and finally calculate the coordinates in the original image in proportion, where the score threshold is 0.5.

[0099] Step S6, filtering the detected sheep's front face according to whether the seven facial feature points of the sheep's front face are detected and whether the detection frame and the seven facial feature points satisfy a positive angle feature relationship;

[0100] Specifically, in step S6, the formula of the positive angle characteristic relationship is as follows:

[0101]

[0102] Where:

[0103] Shape The symbol represents the feature point p m With p n distance; for example, Indicates the distance between feature points p6 and p7;

[0104] Shape The symbol indicates the endpoint of the detection box D m With feature point p n distance; for example Indicates the distance between the detection box endpoint D2 and the feature point p4;

[0105] α1 represents the positive angle facial ratio parameter, which is set to 0.3;

[0106] α2 represents the adaptation parameter between the sheep’s facial frame and facial feature points, which is set to 0.5;

[0107] α3 represents the facial symmetry balance parameter, which is set to 0.2;

[0108] The process of judging the detection result according to the positive angle feature relationship is: whether a rectangular sheep face frame including an upper left endpoint D1, an upper right endpoint D2, a lower left endpoint D3, a lower right endpoint D4 and seven facial feature points including the left corner of the sheep's left eye p1, the right corner of the left eye p2, the left corner of the right eye p3, the right corner of the right eye p4, the left nose p5, the right nose p6 and the nose tip p7 is detected in the image, and whether the relationship between the detection frame endpoints and the seven facial feature points satisfies the positive angle feature relationship.

[0109] If detected, the sheep's face frame will be captured.

[0110] If seven facial feature points are not detected or the formula for the positive angle feature relationship is not satisfied, the detection frame is abandoned.

[0111] The final detection results are all images with 7 facial feature points and the sheep's front face frame marked in the original image.

[0112] Comparative Example 1:

[0113] This comparative example provides a sheep face detection method, which is performed according to the following steps:

[0114] Step S1, collecting a dataset of images of sheep faces of different scales;

[0115] Step S2, marking the sheep face frame and five facial feature points in the dataset image of the sheep's front face to obtain a marked dataset;

[0116] The five facial feature points are the middle point of the left eye, the middle point of the right eye, the left nose wing, the right nose wing and the nose tip;

[0117] Step S3, randomly divide the labeled data set into a training set, a validation set, and a test set;

[0118] Step S4, inputting the training set into the original RetinaFace algorithm for training to obtain a trained algorithm;

[0119] Step S5, pre-process the image to be detected and the key frame of the video and input them into the trained algorithm to obtain the sheep face detection result. The pre-processing process is the same as that in the embodiment.

[0120] Comparative Example 2:

[0121] This comparative example provides a sheep front face detection method. The difference between this method and the embodiment is that this comparative example removes step six in the embodiment, and the other steps are the same as those in the embodiment.

[0122] Experimental verification:

[0123] The detection effect of the efficient multi-scale sheep front face detection method of the present invention and the general detection algorithm on the sheep front face is compared.

[0124] Experimental background: In order to prove the efficiency and accuracy of the present invention in detecting sheep frontal faces, the sheep frontal face detection results of the present invention are compared with those of the original RetinaFace detection algorithm.

[0125] Experimental environment: The operating system is Ubantu18.04.5, and the GPU is NVIDA TITAN RTX / PCle / SSE2.

[0126] Experimental method: using the same test set to compare the sheep front face detection results of the method of the present invention, the method of the present invention given in comparative example 1 using only the original RetinaFace detection algorithm, and the method of the present invention given in comparative example 2 without step S6; the test set includes multi-scale side face and front face pictures of sheep, wherein the multi-scale side face and front face pictures include or do not include sheep front face pictures, and the test set includes 1 / 4, 1 / 10, 1 / 20 three different scales of large, medium and small sheep face pictures, so that the test set pictures are close to the sheep face pictures captured under actual conditions, overcoming the defect that the test set of the existing method contains only a single scale picture, so that the experimental results are as close as possible to the actual sheep face detection conditions; the detection results include the accuracy rate, recall rate and running time of sheep front face detection, and the final results are shown in Table 1, Figure 2 , Figure 3 shown.

[0127] Table 1 Sheep face detection results of different methods

[0128] method Frontal face detection accuracy Frontal face detection recall rate Run time Comparative Example 1 89.34％ 89.87％ 232ms Comparative Example 2 90.65％ 88.31％ 57ms Embodiments of the present invention 95.71％ 87.42％ 63ms

[0129] Result analysis:

[0130] As shown in Table 1, when the traditional detection algorithm, i.e., the original RetinaFace algorithm, is used for multi-scale test set detection, the detection takes a long time, and the front face detection accuracy is low; when the screening step of step six is ​​not used, the improved algorithm shows a significant improvement in detection efficiency, but the front face detection accuracy is not significantly improved; and when the method of the present invention is used and the efficient improved face and feature point detection algorithm is adopted, the detection efficiency can be effectively improved and the detection time can be greatly reduced, and the screening strategy proposed by the present invention can be adopted to greatly improve the front face detection accuracy; however, due to the strict screening conditions, the front faces of sheep in some pictures are missed, but compared with the original RetinaFace algorithm, the missed detection rate of the method of the present invention is not significantly reduced, which is acceptable.

[0131] like Figure 2 As shown, the test picture is a side face of a sheep; Picture 1 is the detection result of Comparative Example 1, and the algorithm determines that the picture contains a front face of a sheep; Picture 2 is the detection result of Comparative Example 2, and the method determines that the picture contains a front face of a sheep; Picture 3 is the detection result of an embodiment of the present invention, and the method determines that the picture does not contain a front face of a sheep. Although the detection algorithm can detect the facial frame and 7 feature points, the positional relationship between the facial frame and the feature points does not conform to Formula 7, so it is determined that the sheep face in the picture does not meet the sheep front face feature.

[0132] like Figure 3 As shown, picture 1 is the detection result of the method of the present invention on the front face picture defined in the test set, and the face frame and 7 facial feature points can be detected in the result, so the judgment result is that the picture contains the front face of the sheep, and the sheep face in the detection frame is captured; picture 2 is the detection result of the method of the present invention on the side face picture defined in the test set, and the face frame and 7 facial feature points can be detected in the result, but the positions of the face frame and the feature points do not conform to formula 7, so the judgment result is that the picture does not contain the front face of the sheep; picture 3 is the detection result of the method of the present invention on another side face picture defined in the test set, and the face frame can be detected in the result but the 7 facial feature points cannot be detected. Since the feature point inspection is incomplete, the judgment result is that the picture does not contain the front face of the sheep.

[0133] The final experiment proves that the method of the present invention has high detection accuracy and short running time on a multi-scale test set simulating actual detection scenes. It is an efficient multi-scale sheep face detection method that is suitable for real-time multi-scale detection in actual sheep farm scenes.

[0134] Comparative Example 3:

[0135] This comparative example provides a human face front face detection method, device and electronic device known in the prior art, but the implementation steps of the method do not mention a specific and achievable key point detection method suitable for the method; at the same time, the method uses 21 key points of the human face to perform affine transformation on the original image to generate a comparison image, and classifies the first human face image and the image after affine transformation, and determines whether it is a front face image by analyzing the degree of distortion of the image; this method is different from step six of the present invention, which is a method for screening the detected front face of the sheep based on whether the 7 feature points of the front face of the sheep are detected and whether the detection frame and the feature points satisfy the positive angle feature relationship; this comparative example cannot effectively detect the front face of the sheep.

[0136] Comparative Example 4:

[0137] This comparative example provides a method for detecting frontal face images, in which the cascade classifier and ASM method used in the method respectively detect the distribution of the face area and key points in the image, and only the influence of the angle relationship and symmetry relationship between the key points on the frontal face screening is considered during screening. This comparative example cannot effectively detect the frontal face of sheep.

[0138] Compared with comparative example 4, the method of the present invention targets the characteristic points specific to sheep and uses the model obtained by deep learning to simultaneously predict the sheep face rectangular frame and facial characteristic points in the image with multi-scale and multi-target efficient detection characteristics, and can avoid the key point deviation caused by multiple predictions; at the same time, the method of the present invention comprehensively considers the facial frame and the characteristic points, and screens the positive angle facial proportion, the adaptation ratio of the facial frame and the characteristic points, and the facial symmetry balance; and uses the adaptation of the facial frame and the characteristic points as a strong determining factor, which is more prominent than other determining factors.

Claims

1. An efficient multi-scale sheep face detection method, characterized in that: The method proceeds as follows: Step S1, collecting a dataset of images of sheep faces of different scales; Step S2, marking the sheep face frame and seven facial feature points in the dataset image of the sheep's front face to obtain a marked dataset; The seven facial feature points are the left eye corner point of the left eye, the right eye corner point of the left eye, the left eye corner point of the right eye, the right eye corner point of the right eye, the left nose wing, the right nose wing and the nose tip; Step S3, randomly divide the labeled data set into a training set, a validation set, and a test set; Step S4, inputting the training set into an efficient face and feature point detection network for training to obtain a trained network model; The efficient face and feature point detection network is an improved RetinaFace algorithm; In step S4, the improved method of the RetinaFace algorithm comprises the following steps: Step S401, replacing the backbone network of the original RetinaFace algorithm with an improved lightweight backbone network MobileNetV3 large, wherein the improvement process of the lightweight backbone network MobileNetV3 large is as follows: All 3 in the lightweight backbone network MobileNetV3 large 3. The layer-by-layer convolution in the depth-wise separable convolution is replaced with switchable dilated convolution, and the dilation rate of the dilated convolution is adjusted through the switch operation during the training process to achieve soft switching of the convolution calculation, so that the lightweight backbone network MobileNetV3 large can obtain receptive fields of different scales, which is conducive to multi-scale detection; The Switch operation is composed of a 5*5 average pooling layer and a 1*1 ordinary convolution layer; Step S402, adjusting the attention module configuration scheme in the network architecture of the lightweight backbone network MobileNetV3 large; The adjustment method is as follows: adding an attention mechanism module to the 1st to 6th layers and the 11th to 15th layers of the bneck module in the lightweight backbone network MobileNetV3 large; The attention mechanism module is composed of a global pooling layer, a fully connected layer, a ReLU function, a fully connected layer and a Sigmoid function; Step S403, adding a channel strengthening module to the three-layer context modules of the SSH structure in the original RetinaFace algorithm; The channel enhancement module consists of two parts, the first part consists of a horizontal average pooling layer, a 1*1 ordinary convolution layer, a 1*1 ordinary convolution layer, a normalization layer, a nonlinear layer, a 1*1 ordinary convolution layer and a Sigmoid function; the second part consists of a vertical average pooling layer, a 1*1 ordinary convolution layer shared with the first part, a normalization layer and a nonlinear layer shared with the first part, a 1*1 ordinary convolution layer, a 1*1 ordinary convolution layer and a Sigmoid function; Step S404, optimize the loss function in the original RetinaFace algorithm, and the final loss function is shown as follows: Where: is the final loss function; i is the anchor frame; is the classification loss function, which is composed of the softmax binary classification loss function; is the predicted anchor box i The probability that contains a sheep face; is the actual anchor box i The probability of containing a sheep face, which takes a value of 1 or 0. A value of 1 means that the actual frame contains a sheep face, and a value of 0 means that the actual frame does not contain a sheep face; is the detection box position loss function, Function composition; is the prediction parameter of the detection box position loss function, including the predicted anchor box i The center coordinates of , , and the width and height of anchor box i , ; is the actual parameter of the detection box position loss function, including the actual annotation box i The center coordinates of , , and width and height values , ; To detect facial feature points, the loss function is: Function composition; To detect the predicted parameters of the facial feature point loss function, including the predicted coordinate values ​​of the seven facial feature point coordinates ; To detect the actual parameters of the facial feature point loss function, including the coordinate values ​​of the seven facial feature points actually marked ; is 0.5; is 0.2; Step S5, pre-processing the image and video key frame to be detected and inputting them into the trained network model to obtain the sheep face detection result; The sheep face detection result includes 7 facial feature points and a detection frame of the sheep's front face; Step S6, filtering the detected sheep's front face according to whether the seven facial feature points of the sheep's front face are detected and whether the detection frame and the seven facial feature points satisfy a positive angle feature relationship; The formula for the positive angle characteristic relationship is as follows, Where: Shape The symbol represents the feature point and distance; Shape The symbol represents the endpoint of the detection box With feature points distance; represents the positive angle facial proportion parameter; Indicates the adaptation parameters of the sheep's face frame and facial feature points; Represents facial symmetry balance parameters.

2. The efficient multi-scale sheep face detection method according to claim 1, characterized in that: Step S1 specifically includes the following steps: Step S101, using a camera to shoot a video of sheep in a farm. When shooting the video of the sheep, the front face of the sheep must be included. In the video, the sheep's face is located in different positions of the image instead of a single middle position. The sheep's face is distributed in five positions of the upper left, lower left, upper right, lower right and middle of the original image; and the collected image contains sheep faces of multiple scales; The multi-scale is divided into three types: large scale, medium scale and small scale; the large scale accounts for 1 / 4 of the image ratio; the medium scale accounts for 1 / 10 of the image ratio; the small scale accounts for 1 / 20 of the image ratio; Step S102, processing the video of the sheep, capturing key frames in the video, saving an image containing all features of the complete front face of the sheep, and obtaining a dataset image of the front face of the sheep; All the features of the sheep's front face include the features of the left corner and the right corner of the left eye, the features of the left corner and the right corner of the right eye, the features of the left nose, the features of the right nose and the features of the nose tip.

3. The efficient multi-scale sheep face detection method as claimed in claim 1, characterized in that: Step S2 specifically includes the following steps: Step S201, annotating the dataset image of the sheep's front face, including annotating a square sheep face frame and seven facial feature points; In the process of marking the sheep face frame, the coordinates of the upper left point and the lower right point of the frame are recorded; Step S202, import the annotated coordinates into a text file, each picture record in the text file contains two lines of information; the first line starts with # and contains the absolute path of the picture; the second line contains the upper left coordinates, lower right coordinates, left eye corner coordinates, left eye right eye corner coordinates, right eye left eye corner coordinates, right eye right eye corner coordinates, left nose wing point coordinates, right nose wing point coordinates and nose tip point coordinates of the sheep face frame; the coordinates are stored according to x and y, and each coordinate point is separated by 1 or 0, 1 indicates that the point has high credibility and needs to be adopted, and 0 indicates that the credibility is low and should not be adopted, so as to obtain a labeled data set.

4. The efficient multi-scale sheep face detection method as claimed in claim 1, characterized in that: In step S4, the accuracy and recall rate of the training model are tested on the validation set every 5 rounds of training to verify the model detection effect; the final model detection accuracy of the trained model is tested on the test set, and the model is considered acceptable if and only if the model accuracy is higher than 95%.

5. The efficient multi-scale sheep face detection method as claimed in claim 1, characterized in that: In step S5, the preprocessing is: scaling the width of the image to 320 pixels and compressing the height of the image proportionally according to the original image ratio.

6. The efficient multi-scale sheep face detection method according to claim 1, characterized in that: In step S6, Set to 0.3; Set to 0.5; Set to 0.

2.

7. The efficient multi-scale sheep face detection method as claimed in claim 1, characterized in that: In step S6, the process of judging the detection result according to the positive angle characteristic relationship is as follows: whether a rectangular sheep face frame including the upper left end point is detected in the image , upper right endpoint , lower left endpoint , lower right endpoint and including the left corner of the sheep's left eye and the left and right corners of the eyes , right eye left corner and the right corner of the right eye , left nasal cavity , right nasal profile and nose tip There are 7 facial feature points in total, and the relationship between the detection frame endpoints and the 7 facial feature points satisfies the positive angle feature relationship; If detected, the sheep’s face frame will be captured; If 7 facial feature points are not detected or the formula of the positive angle feature relationship is not satisfied, the detection frame is abandoned; The final detection results are all images with 7 facial feature points and the sheep's front face frame marked in the original image.

Images