Cattle face recognition method, device and equipment and storage medium

Abstract

The present application relates to artificial intelligence, and provides a cattle face recognition method, device, equipment and storage medium. The method filters the face image to obtain an initial image including a front face image, filters a cattle head region image extracted from the initial image to obtain a target cattle head image and aligns to obtain an aligned image, instance segments the front face image to obtain an instance image, augments the aligned image and the instance image, fuses face features of a first augmented image and local features of a second augmented image, obtains target features and performs object recognition to obtain an object identity, and improves recognition accuracy. In addition, the present application also relates to blockchain technology, and the object identity can be stored in the blockchain.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a cattle face recognition method, device and equipment and storage medium. BACKGROUND

[0002] With the development of artificial intelligence, a cattle face recognition scheme has also emerged. In the current cattle face recognition scheme based on artificial intelligence, due to the randomness of cattle face images, it is difficult to accurately obtain the key points of the cattle face images, resulting in the inability to accurately identify specific objects in the cattle face images. SUMMARY

[0003] In view of the above, it is necessary to provide a cattle face recognition method, device, equipment and storage medium, which can solve the technical problem of being unable to accurately identify specific objects in cattle face images.

[0004] In one aspect, the present application provides a cattle face recognition method, which comprises:

[0005] Obtaining face images of a collection object in different directions;

[0006] Based on key point detection and quality detection of the face images, filtering the face images to obtain initial images, the initial images including front face images of the collection object;

[0007] Based on quality detection of cattle head region images obtained by semantic segmentation from the initial images, filtering the cattle head region images to obtain target cattle head images;

[0008] Aligning the target cattle head images to obtain aligned images, and performing instance segmentation on the front face images to obtain instance images;

[0009] Performing normalization augmentation processing on the aligned images to obtain first augmented images, and performing normalization augmentation processing on the instance images to obtain second augmented images;

[0010] Fusing face features of the first augmented images and local features of the second augmented images to obtain target features;

[0011] Based on the target features, performing object recognition to obtain the object identity of the collection object.

[0012] According to the preferred embodiment of the present application, the filtering of the face images based on key point detection and quality detection of the face images to obtain initial images comprises:

[0013] Performing key point detection on the face images to obtain face key points;

[0014] Identifying the object category corresponding to the collection object;

[0015] identify a detection region from the face image based on the object category, the object configuration shape and the face key points;

[0016] calculate a pixel variance value of the detection region;

[0017] filter the face image based on a comparison between the pixel variance value and a preset deviation threshold, to obtain a filtered face image;

[0018] calculate an image definition of the filtered face image;

[0019] process the filtered face image based on the image definition, to obtain the initial image.

[0020] According to the preferred embodiment of the present application, the instance segmentation on the frontal face image to obtain an instance image comprises:

[0021] perform target detection on the frontal face image to obtain object center coordinates of the frontal face image;

[0022] obtain an offset threshold of the object category;

[0023] identify an object contour image from the frontal face image based on the object center coordinates and the offset threshold;

[0024] adjust an image shape of the object contour image based on the object configuration shape, to obtain a target contour image;

[0025] perform mask processing on the target contour image, to obtain the instance image.

[0026] According to the preferred embodiment of the present application, the quality detection on the ox head region image obtained from the semantic segmentation on the initial image, to filter the ox head region image, to obtain a target ox head image comprises:

[0027] identify a pixel type to which each pixel point in the initial image belongs, based on a pixel value and a pixel position of each pixel point in the initial image;

[0028] determine a region formed by pixel points of the preset type as the ox head region image;

[0029] calculate proportion information of a pixel number in the ox head region image to an image area of the ox head region image;

[0030] filter the ox head region image according to the proportion information and a preset proportion threshold, and determine a filtered ox head region image as the target ox head image.

[0031] According to a preferred embodiment of the present application, the aligning the target cattle head image to obtain an aligned image comprises:

[0032] Identifying a key point pair from the plurality of target cattle head images;

[0033] Calculating an alignment transformation relationship of the key point pair according to coordinate information of the key point pair in the plurality of target cattle head images;

[0034] Performing alignment processing on the plurality of target cattle head images based on the alignment transformation relationship to obtain the aligned image.

[0035] According to a preferred embodiment of the present application, the fusing the face feature of the first augmented image and the local feature of the second augmented image to obtain a target feature comprises:

[0036] Extracting the face feature from the first augmented image based on a first feature extraction model trained in advance, the face feature comprising a face feature element at each matrix position;

[0037] Extracting the local feature from the second augmented image based on a second feature extraction model trained in advance, the local feature comprising a local feature element at each matrix position;

[0038] Performing voting processing on the face feature element and the corresponding local feature element at each matrix position to obtain a target element corresponding to each matrix position;

[0039] Splicing the target elements at a plurality of matrix positions to obtain the target feature.

[0040] According to a preferred embodiment of the present application, before extracting the face feature from the first augmented image based on the first feature extraction model trained in advance, the method further comprises:

[0041] Obtaining a first training image and a first labeled feature of a first training object, and obtaining a second training image and a second labeled feature of a second training object;

[0042] Extracting a feature from the first training image based on a feature extraction network constructed in advance to obtain a predicted feature;

[0043] Adjusting parameters of the feature extraction network based on a first feature distance between the predicted feature and the first labeled feature, and a second feature distance between the predicted feature and the second labeled feature, until the first feature distance is minimum and the second feature distance is maximum, to obtain the first feature extraction model.

[0044] In another aspect, the present application further provides a cattle face recognition device, which comprises:

[0045] an acquisition unit, configured to acquire face images of a collection object in different orientations;

[0046] a filtering unit, configured to filter the face images based on key point detection and quality detection on the face images, to obtain initial images, the initial images including frontal face images of the collection object;

[0047] The filtering unit is further configured to filter the ox head region images based on quality detection on the ox head region images obtained through semantic segmentation from the initial images, to obtain target ox head images.

[0048] an alignment and segmentation unit, configured to align the target ox head images to obtain aligned images, and to perform instance segmentation on the frontal face images to obtain instance images;

[0049] an augmentation unit, configured to perform normalization augmentation processing on the aligned images to obtain first augmented images, and to perform normalization augmentation processing on the instance images to obtain second augmented images;

[0050] a fusion unit, configured to fuse face features of the first augmented images and local features of the second augmented images to obtain target features;

[0051] a recognition unit, configured to perform object recognition based on the target features to obtain an object identity of the collection object.

[0052] In another aspect, the present application further provides an electronic device, which comprises:

[0053] a memory, configured to store computer readable instructions; and

[0054] a processor, configured to execute the computer readable instructions stored in the memory to implement the ox face recognition method.

[0055] In another aspect, the present application further provides a computer readable storage medium, which stores computer readable instructions, and the computer readable instructions are executed by a processor in an electronic device to implement the ox face recognition method.

[0056] From the above technical scheme can be seen, the application can improve the image quality of the initial image and the target cattle head image by detecting the quality of the face image and the quality of the cattle head region image extracted from the initial image, effectively avoiding the problem of image quality that leads to inaccurate recognition of the object identity. Furthermore, the application combines the face features and local features of the collection object corresponding to the different orientations, which can fully consider the features of the collection object and effectively improve the robustness of cattle face recognition. In addition, the application can avoid the problem of image angle that leads to inaccurate recognition of the object identity during object recognition by performing augmentation processing on the aligned image and the instance image, thereby further improving the accuracy of cattle face recognition. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a preferred embodiment of the application of the cattle face recognition method.

[0058] Figure 2 is a flowchart of a preferred embodiment of the cattle face recognition method of the application.

[0059] Figure 3 is a schematic diagram of the object configuration shape in the cattle face recognition method of the application.

[0060] Figure 4 is a functional module diagram of a preferred embodiment of the cattle face recognition device of the application.

[0061] Figure 5 is a structural schematic diagram of an electronic device for implementing a preferred embodiment of the cattle face recognition method of the application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be described in detail below with reference to the drawings and specific embodiments.

[0063] As shown in Figure 1 is a preferred embodiment of the application of the cattle face recognition method. The cattle face recognition method can be applied to one or more electronic devices 1, and the electronic device 1 communicates with a camera device 2, a camera device 3 and a camera device 4. The camera device 2, the camera device 3 and the camera device 4 can be any device capable of shooting. The camera device 2, the camera device 3 and the camera device 4 are respectively located at different orientation positions to realize shooting of the collection object at different angles.

[0064] The cow face recognition method can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0065] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0066] The cow face recognition method is applied to one or more electronic devices. The electronic device is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored computer readable instructions. The hardware thereof includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0067] The electronic device can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smartphone, a personal digital assistant (PDA), a game console, an interactive Internet Protocol Television (IPTV), a smart wearable device, etc.

[0068] The electronic device can include a network device and / or a user device. The network device includes, but is not limited to, a single network electronic device, a group of electronic devices composed of multiple network electronic devices, or a cloud composed of a large number of hosts or network electronic devices based on cloud computing.

[0069] The network in which the electronic device is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0070] For example, the cow face recognition method can be implemented by a computer program product, which includes a computer program and a computer readable medium. The computer readable medium stores the computer program, and the computer program is used to instruct the computer to perform the cow face recognition method. Figure 2Fig. 1 shows a flow chart of a preferred embodiment of the cattle face recognition method. The order of the steps in the flow chart can be changed according to different requirements, and some steps can be omitted. The subject performing the method is an electronic device, such as Figure 1 Fig. 1 shows an electronic device 1.

[0071] 101, obtaining face images of a collection object in different orientations.

[0072] In at least one embodiment of the present application, the collection object can refer to a specific object that needs to be face-recognized, for example, the collection object can be a cattle numbered 001.

[0073] The different orientations can be left, front, right, and the like.

[0074] In at least one embodiment of the present application, the electronic device controls the camera devices installed in different positions to take pictures of the collection object, thereby obtaining the face images.

[0075] In at least one embodiment of the present application, the face images can be obtained from a storage device connected to the plurality of camera devices.

[0076] The storage device can simultaneously communicate with the plurality of camera devices.

[0077] 102, filtering the face images based on key point detection and quality detection of the face images, thereby obtaining initial images, the initial images including front face images of the collection object.

[0078] In at least one embodiment of the present application, the initial images refer to images obtained after deleting face images that have face occlusion problems, cattle head problems, and image blur problems. The initial images contain cattle head and other image information. For example, the other image information can be scene, cattle body, and the like.

[0079] The front face images refer to images obtained by a camera device in front of the collection object.

[0080] In at least one embodiment of the present application, the electronic device filters the face images based on key point detection and quality detection of the face images, thereby obtaining initial images, including:

[0081] Key point detection is performed on the face images to obtain face key points.

[0082] Identifying the object category corresponding to the collection object.

[0083] identify a detection region from the face image based on the object configuration shape of the object category and the face key point;

[0084] calculate a pixel variance value of the detection region;

[0085] filter the face image based on a comparison between the pixel variance value and a preset deviation threshold, to obtain a filtered face image;

[0086] calculate an image definition of the filtered face image;

[0087] process the filtered face image based on the image definition, to obtain the initial image.

[0088] The face key point can be an eye, a mouth, or the like.

[0089] The object category refers to a specific category to which the collected object belongs. For example, the object category can be a cow.

[0090] The object configuration shape refers to a face shape corresponding to the object category obtained from an Internet database. For example, Figure 3 Figure 3 is a schematic diagram of the object configuration shape in the cow face recognition method of the present application, Figure 3 which is only an example.

[0091] The detection region refers to a region image containing the face key point in the face image, and the region shape of the detection region is the same as the object configuration shape. Specifically, the electronic device identifies a detection region from the face image based on the object configuration shape of the object category and the face key point, which includes:

[0092] obtain a configuration image corresponding to the object configuration shape, and identify a configuration key point in the configuration image corresponding to the face key point;

[0093] extract a region corresponding to the configuration image from the face image as the detection region based on a mapping relationship between the configuration key point and the face key point.

[0094] The pixel variance value refers to variance information of pixel values of all pixel points in the detection region, and the pixel variance value is used to measure the discrete degree of each pixel point from all pixel points.

[0095] The preset deviation threshold can be set according to actual needs. For example, the preset deviation threshold can be 0.5.

[0096] The image definition is used to measure the definition degree of the filtered face image. ​

[0097] By combining the object configuration shape and the pixel variance value, a face image with a face occlusion problem and / or a partial face problem can be identified and filtered, and by identifying the image definition of the filtered face image, the definition of the initial image can be ensured, thereby facilitating improvement of the robustness of the cattle face recognition.

[0098] Specifically, the electronic device filters the face image based on a comparison of the pixel variance value and a preset deviation threshold, and obtains a filtered face image, including:

[0099] If the pixel variance value is greater than or equal to the preset deviation threshold, the face image is rejected, and the filtered face image is obtained.

[0100] In this embodiment, when the pixel variance value is greater than or equal to the preset deviation threshold, it indicates that the face image has a face occlusion problem and / or a partial face problem, and therefore, by rejecting the face image, the quality of the filtered face image can be improved.

[0101] Specifically, the calculation formula of the image definition is:

[0102]

[0103]

[0104]

[0105] wherein D represents the image definition, K1 represents the pixel variance of the filtered face image, K2 represents the pixel difference value of the filtered face image, W represents the image width of the filtered face image, the pixel point number corresponding to the image width is [0, i+n-1], H represents the image length of the filtered face image, the pixel point number corresponding to the image length is [0, j+n-1], x i+n represents the pixel value of the i+nth pixel point in each row of the filtered face image, x i represents the pixel value of the ith pixel point in each row of the filtered face image, y j+n represents the pixel value of the j+nth pixel point in each column of the filtered face image, y j represents the pixel value of the jth pixel point in each column of the filtered face image, and n represents a positive integer value other than 0.

[0106] Specifically, the electronic device processes the filtered face image based on the image definition, and obtains the initial image, including:

[0107] The filtered facial image with the image definition greater than the preset definition threshold is determined as the initial image.

[0108] The preset definition threshold can be set according to actual requirements.

[0109] 103. Based on quality detection of the ox head region image obtained by semantic segmentation from the initial image, the ox head region image is filtered to obtain a target ox head image.

[0110] In at least one embodiment of the present application, the ox head region image refers to a region image containing an ox head in the initial image.

[0111] The target ox head image refers to an ox head region image with proportion information greater than a preset proportion threshold. The target ox head image includes a left face image, a front face image, a right face image, etc. The proportion information is used to measure the definition of the ox head region image, and the greater the proportion information, the higher the definition of the ox head region image.

[0112] In at least one embodiment of the present application, the electronic device filters the ox head region image based on quality detection of the ox head region image extracted from the initial image to obtain a target ox head image, which includes:

[0113] Based on the pixel value and pixel position of each pixel point in the initial image, the pixel type to which each pixel point in the initial image belongs is identified.

[0114] The region formed by the pixel points with the preset type is determined as the ox head region image.

[0115] The proportion information of the number of pixels in the ox head region image to the image area of the ox head region image is calculated.

[0116] According to the proportion information and the preset proportion threshold, the ox head region image is filtered, and the filtered ox head region image is determined as the target ox head image.

[0117] The pixel type can include, but is not limited to, ox head, scene, ox tail, etc.

[0118] The preset type is usually set as ox head.

[0119] The preset proportion threshold can be set according to actual requirements.

[0120] By combining the pixel value and the pixel position, the pixel type to which each pixel point belongs can be accurately identified, so as to improve the extraction accuracy of the cow head region image. By combining the number of pixels in the cow head region image and the image area, the proportion information for measuring the definition of the cow head region image can be accurately generated. Then, by comparing the proportion information with the preset proportion threshold, the target cow head image can be reasonably screened, so as to ensure the quality of the target cow head image.

[0121] Specifically, the electronic device identifies the pixel type to which each pixel point in the initial image belongs based on the pixel value and the pixel position of each pixel point in the initial image.

[0122] An initial type corresponding to the pixel value is obtained from a preset template library.

[0123] If there are multiple initial types, a mapping type corresponding to the pixel position is obtained from the multiple initial types as the pixel type.

[0124] The preset template library stores a mapping relationship among multiple pixel values, multiple pixel positions, and multiple pixel types.

[0125] By combining the pixel value and the pixel position, the pixel type can be accurately identified.

[0126] Specifically, the electronic device filters the cow head region image according to the proportion information and a preset proportion threshold, and determines the filtered cow head region image as the target cow head image.

[0127] The cow head region whose proportion information is less than the preset proportion threshold is removed, and the remaining cow head region image is determined as the target cow head image.

[0128] 104. Align the target cow head image to obtain an aligned image, and perform instance segmentation on the frontal face image to obtain an instance image.

[0129] In at least one embodiment of the present application, the aligned image refers to an image generated after direction registration of multiple target cow head images.

[0130] In at least one embodiment of the present application, the electronic device aligns the target cow head image to obtain an aligned image.

[0131] Identify a key point pair from multiple target cow head images.

[0132] According to the coordinate information of the key point pair in the multiple target cow head images, the alignment transformation relationship of the key point pair is calculated.

[0133] Based on the alignment transformation relationship, the plurality of target cattle head images are processed to obtain the alignment image.

[0134] The key point pair refers to the key point identified from the plurality of target cattle head images, for example, the key point pair can be based on the feature point of the eye, the eye corner.

[0135] The calculation formula of the alignment transformation relationship is:

[0136]

[0137]

[0138] q∈(1,z), p∈(1,z), q≠p;

[0139] Wherein, The alignment transformation relationship is represented by T1, T2, …, T k , T z×(z-1) / 2 , and the transformation relationship of any two key points in the key point pair is represented by (x q , y q ) and (x p , y p ), wherein (x q , y q ) and (x p , y p ) respectively represent the coordinate information of any two key points in the key point pair in the plurality of target cattle head images, and z represents the total number of key points in the key point pair.

[0140] In at least one embodiment of the present application, the electronic device performs instance segmentation on the front face image to obtain an instance image, including:

[0141] Target detection is performed on the front face image to obtain the object center coordinates of the front face image.

[0142] An offset threshold of the object category is obtained.

[0143] Based on the object center coordinates and the offset threshold, an object contour image is identified from the front face image.

[0144] Based on the object configuration shape, the image shape of the object contour image is adjusted to obtain a target contour image.

[0145] The target contour image is subjected to mask processing to obtain the instance image.

[0146] The object center coordinates refer to the coordinate value corresponding to the center position of the object in the front face image.

[0147] The offset threshold can be set according to the maximum value of the distance between the head center and the face edge in all objects corresponding to the object category in actual life. The offset threshold can be multiple, for example, the offset threshold includes a threshold value of the distance between the head center and the left face edge, a threshold value of the distance between the head center and the right face edge, and the like.

[0148] The image shape of the target contour image is the same as the object configuration shape.

[0149] Specifically, the electronic device identifies an object contour image from the front face image based on the object center coordinate and the offset threshold, including:

[0150] The sum of the object center coordinate and the offset threshold is calculated to obtain a contour coordinate;

[0151] An area image constituted by the contour coordinate is obtained from the front face image as the object contour image.

[0152] By combining the offset threshold, the contour coordinate can be accurately obtained, so that the object contour image is reasonably determined.

[0153] 105. The aligned image is normalized and augmented to obtain a first augmented image, and the instance image is normalized and augmented to obtain a second augmented image.

[0154] In at least one embodiment of the present application, the first augmented image refers to an image obtained after pixel value normalization, angle rotation and / or size transformation of the aligned image.

[0155] The second augmented image refers to an image obtained after pixel value normalization, angle rotation and / or size transformation of the instance image.

[0156] 106. The face features of the first augmented image and the local features of the second augmented image are fused to obtain target features.

[0157] In at least one embodiment of the present application, the face features refer to global features extracted from the first augmented image by a first feature extraction model trained in advance.

[0158] The local features refer to feature information extracted from the second augmented image by a second feature extraction model trained in advance.

[0159] The target features refer to fusion features obtained after voting processing of the face features and the local features.

[0160] In at least one embodiment of the present application, the electronic device fuses the facial features of the first augmented image and the local features of the second augmented image to obtain target features, including:

[0161] Based on a pre-trained first feature extraction model, the facial features are extracted from the first augmented image, and the facial features include facial feature elements at each matrix position;

[0162] Based on a pre-trained second feature extraction model, the local features are extracted from the second augmented image, and the local features include local feature elements at each matrix position;

[0163] The facial feature elements and the corresponding local feature elements at each matrix position are subjected to voting processing to obtain target elements corresponding to each matrix position;

[0164] The target elements are spliced based on a plurality of matrix positions to obtain the target features.

[0165] Wherein, the target element refers to the element with the largest number of values among the facial feature elements and the corresponding local feature elements, for example, the facial feature elements include 1, 1, 1, 0, 0, and the corresponding local feature elements are 1. Since the element 1 corresponds to 4 values and the element 0 corresponds to 2 values, the target element is 1.

[0166] By voting processing the facial feature elements and the corresponding local feature elements at each matrix position, the determination accuracy of the target elements can be improved, and thus the target elements are spliced based on the plurality of matrix positions to improve the fusion accuracy of the target features.

[0167] In at least one embodiment of the present application, before extracting the facial features from the first augmented image based on the pre-trained first feature extraction model, the method further includes:

[0168] Obtaining a first training image and a first labeled feature of a first training object, and obtaining a second training image and a second labeled feature of a second training object;

[0169] Performing feature extraction on the first training image based on a pre-constructed feature extraction network to obtain predicted features;

[0170] Based on a first feature distance between the predicted features and the first labeled features, and a second feature distance between the predicted features and the second labeled features, the parameter of the feature extraction network is adjusted until the first feature distance is minimum and the second feature distance is maximum, and the first feature extraction model is obtained.

[0171] Wherein, the first training object and the second training object are not the same object. For example, the first training object is a cow with the number 003, and the second training object is a cow with the number 004.

[0172] The feature extraction network includes multiple convolutional layers, and the parameters of the feature extraction network include the size of each convolutional kernel, etc.

[0173] By combining the feature distances of the same training object and different training objects, the parameters of the feature extraction network are adjusted to improve the feature extraction capability of the first feature extraction model.

[0174] In other embodiments, the electronic device may further adjust the parameters of the feature extraction network using the metric learning loss values ​​of the first training image and the second training image until the metric learning loss value is minimized, thereby obtaining the feature extraction model. The formula for calculating the metric learning loss value is:

[0175]

[0176]

[0177]

[0178] Where L represents the metric learning loss value, N represents the number of samples in the first training image and the second training image, M represents the number of categories, m is the angular margin, m∈[0.1,8], θ∈[0,π-m], x t y represents the features extracted by the feature extraction network from the t-th training image. t This represents the category number corresponding to the t-th training image. Indicates feature x t With the y-th matrix weight t The angle between column vectors, W g The g-th column represents the matrix weights, which are network parameters in the feature extraction network.

[0179] This embodiment enables the accurate calculation of the metric learning loss value, thereby improving the adjustment accuracy of the feature extraction model.

[0180] In at least one embodiment of the present invention, the training method of the second feature extraction model is similar to that of the first feature extraction model, and will not be described in detail here.

[0181] 107. Based on the target features, perform object recognition to obtain the object identity of the collected object.

[0182] It is emphasized that, in order to further ensure the privacy and security of the object identity, the object identity can also be stored in a node of a block chain.

[0183] In at least one embodiment of the present application, the object identity refers to unique identification information of the collection object, for example, the object identity can be an identity number of the collection object and the like.

[0184] In at least one embodiment of the present application, the electronic device performs object recognition based on the target feature to obtain the object identity of the collection object, which includes:

[0185] Matching the target feature with all feature templates in a preset farm to obtain a matching degree;

[0186] Determining the object information corresponding to the feature template with the largest matching degree as the object identity.

[0187] The preset farm can be a farm that needs to perform cow face recognition.

[0188] As can be seen from the above technical solution, the present application can improve the image quality of the initial image and the target cow head image through quality detection of the face image and quality detection of the cow head region image extracted from the initial image, effectively avoiding the problem that the object identity cannot be accurately recognized due to image quality problems, and further, the present application can fully consider the features of the collection object by combining the face features and local features of the collection object in multiple different orientations, effectively improving the robustness of cow face recognition. In addition, the present application can avoid the problem that the object identity cannot be accurately recognized due to image angle and the like during object recognition by performing augmentation processing on the aligned image and the instance image, thereby further improving the accuracy of cow face recognition.

[0189] As Figure 4 shown is a functional module diagram of a preferred embodiment of a cow face recognition device of the present application. The cow face recognition device 11 includes an acquisition unit 110, a filtering unit 111, an alignment and segmentation unit 112, an augmentation unit 113, a fusion unit 114, an identification unit 115, an extraction unit 116, and an adjustment unit 117. The module / unit referred to in the present application refers to a series of computer readable instructions that can be acquired by the processor 13 and can complete a fixed function, which is stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in the subsequent embodiments.

[0190] The acquisition unit 110 is configured to acquire face images of a collection object in different orientations.

[0191] The filtering unit 111 is configured to filter the face image based on key point detection and quality detection of the face image, to obtain an initial image, wherein the initial image comprises a frontal face image of the collection object.

[0192] The filtering unit 111 is further configured to filter the ox head region image based on quality detection of the ox head region image obtained through semantic segmentation from the initial image, to obtain a target ox head image.

[0193] The alignment and segmentation unit 112 is configured to align the target ox head image to obtain an aligned image, and perform instance segmentation on the frontal face image to obtain an instance image.

[0194] The augmentation unit 113 is configured to perform normalization augmentation processing on the aligned image to obtain a first augmented image, and perform normalization augmentation processing on the instance image to obtain a second augmented image.

[0195] The fusion unit 114 is configured to fuse face features of the first augmented image and local features of the second augmented image to obtain target features.

[0196] The recognition unit 115 is configured to perform object recognition based on the target features to obtain an object identity of the collection object.

[0197] In at least one embodiment of the present application, the filtering unit 111 is further configured to perform key point detection on the face image to obtain face key points.

[0198] Recognize the object category corresponding to the collection object.

[0199] Based on the object configuration shape of the object category and the face key points, a detection region is recognized from the face image.

[0200] Calculate the pixel variance value of the detection region.

[0201] Based on comparison of the pixel variance value with a preset deviation threshold, filter the face image to obtain a filtered face image.

[0202] Calculate the image sharpness of the filtered face image.

[0203] Process the filtered face image based on the image sharpness to obtain the initial image.

[0204] In at least one embodiment of the present application, the filtering unit 111 is further configured to filter the ox head region image based on quality detection of the ox head region image extracted from the initial image, to obtain a target ox head image, comprising:

[0205] identify a pixel type to which each pixel point in the initial image belongs based on a pixel value and a pixel position of each pixel point in the initial image;

[0206] determine a region formed by the pixel points of the preset type as the cow head region image;

[0207] calculate proportion information of a number of pixels in the cow head region image to an image area of the cow head region image;

[0208] filter the cow head region image according to the proportion information and a preset proportion threshold, and determine a filtered cow head region image as the target cow head image.

[0209] In at least one embodiment of the present application, the alignment and segmentation unit 112 is further configured to align the target cow head image to obtain an aligned image, including:

[0210] identify a key point pair from a plurality of target cow head images;

[0211] calculate an alignment transformation relationship of the key point pair based on coordinate information of the key point pair in the plurality of target cow head images;

[0212] align the plurality of target cow head images based on the alignment transformation relationship to obtain the aligned image.

[0213] In at least one embodiment of the present application, the alignment and segmentation unit 112 is further configured to perform target detection on the frontal face image to obtain an object center coordinate of the frontal face image;

[0214] obtain an offset threshold of the object category;

[0215] identify an object contour image from the frontal face image based on the object center coordinate and the offset threshold;

[0216] adjust an image shape of the object contour image based on the object configuration shape to obtain a target contour image;

[0217] perform mask processing on the target contour image to obtain the instance image.

[0218] In at least one embodiment of the present application, the fusion unit 114 is further configured to extract the face feature from the first augmented image based on a first feature extraction model trained in advance, the face feature including a face feature element of each matrix position;

[0219] extract the local feature from the second augmented image based on a second feature extraction model trained in advance, the local feature including a local feature element of each matrix position;

[0220] Voting is performed on the face feature elements and the corresponding local feature elements at each matrix position to obtain a target element corresponding to each matrix position;

[0221] The target features are spliced based on the plurality of matrix positions to obtain the target features.

[0222] In at least one embodiment of the present application, the acquisition unit 110 is further configured to acquire a first training image and a first labeled feature of a first training object, and acquire a second training image and a second labeled feature of a second training object;

[0223] The extraction unit 116 is configured to perform feature extraction on the first training image based on a pre-constructed feature extraction network to obtain a predicted feature;

[0224] The adjustment unit 117 is configured to perform parameter adjustment on the feature extraction network based on a first feature distance between the predicted feature and the first labeled feature, and a second feature distance between the predicted feature and the second labeled feature, until the first feature distance is minimized and the second feature distance is maximized, to obtain the first feature extraction model.

[0225] As can be seen from the above technical solutions, the present application can improve the image quality of the initial image and the target cow head image by detecting the quality of the face image and the quality of the cow head region image extracted from the initial image, effectively avoiding the problem of inaccurate recognition of the object identity due to image quality problems, and further, the present application combines the face features and local features of the collection object corresponding to the plurality of different orientations, which can fully consider the features of the collection object, effectively improving the robustness of cow face recognition. In addition, the present application can avoid the problem of inaccurate recognition of the object identity due to image angle and other problems during object recognition by performing augmentation processing on the aligned image and the instance image, thereby further improving the accuracy of cow face recognition.

[0226] As Figure 5 shown is a structural schematic diagram of an electronic device for implementing the preferred embodiment of the cow face recognition method of the present application.

[0227] In one embodiment of the present application, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer readable instructions stored in the memory 12 and executable on the processor 13, such as a cow face recognition program.

[0228] Those skilled in the art can understand that the schematic diagram is only an example of the electronic device 1, and does not constitute a limitation on the electronic device 1, and can include more or less components than the diagram, or combine certain components, or different components, for example, the electronic device 1 can also include an input / output device, a network access device, a bus, etc.

[0229] The processor 13 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor 13 is the operation core and control center of the electronic device 1, and connects various parts of the entire electronic device 1 through various interfaces and lines, and executes the operating system and various installed application programs, program codes, etc. of the electronic device 1.

[0230] For example, the computer readable instructions can be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units can be a series of computer readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer readable instructions in the electronic device 1. For example, the computer readable instructions can be divided into an acquisition unit 110, a filtering unit 111, an alignment and segmentation unit 112, an augmentation unit 113, a fusion unit 114, an identification unit 115, an extraction unit 116, and an adjustment unit 117.

[0231] The memory 12 can be used to store the computer readable instructions and / or modules, and the processor 13 realizes various functions of the electronic device 1 by running or executing the computer readable instructions and / or modules stored in the memory 12, and calling the data stored in the memory 12. The memory 12 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the electronic device, etc. The memory 12 can include non-volatile and volatile memories, for example: a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash storage device, or other storage devices.

[0232] The memory 12 can be an external memory and / or an internal memory of the electronic device 1. Further, the memory 12 can be a memory having a physical form, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0233] The modules / units integrated in the electronic device 1, if implemented in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by computer readable instructions instructing related hardware, and the computer readable instructions can be stored in a computer readable storage medium. When the computer readable instructions are executed by a processor, the steps of the above-mentioned various method embodiments can be implemented.

[0234] The computer readable instructions include computer readable instruction codes, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer readable instruction codes, recording media, U disks, mobile hard disks, magnetic disks, optical disks, computer memories, read-only memories (ROMs), random access memories (RAMs).

[0235] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism and encryption algorithm. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods, each of which contains the information of a batch of network transactions, for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer and an application service layer.

[0236] In combination Figure 2 The memory 12 in the electronic device 1 stores computer readable instructions implementing a cattle face recognition method, and the processor 13 can execute the computer readable instructions to implement:

[0237] Obtaining face images of a collection object in different orientations;

[0238] Filtering the face images based on quality detection of the face images to obtain initial images, the initial images including front face images of the collection object;

[0239] Filtering the cattle head region images based on quality detection of the cattle head region images extracted from the initial images to obtain target cattle head images;

[0240] Aligning the target cattle head images to obtain aligned images, and performing instance segmentation on the front face images to obtain instance images;

[0241] Performing augmentation processing on the aligned images to obtain first augmented images, and performing augmentation processing on the instance images to obtain second augmented images;

[0242] Fusing face features of the first augmented images and local features of the second augmented images to obtain target features;

[0243] Performing object recognition based on the target features to obtain an object identity of the collection object.

[0244] Specifically, the specific implementation method of the processor 13 on the above computer readable instructions can refer to the description of related steps in the corresponding embodiments, which will not be described here. Figure 2

[0245] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There can be another division way in actual implementation.

[0246] ​The computer readable storage medium stores computer readable instructions, wherein the computer readable instructions are executed by the processor 13 to implement the following steps:

[0247] Obtaining face images of the collection object in different orientations;

[0248] Filtering the face images based on quality detection of the face images to obtain initial images, wherein the initial images include front face images of the collection object;

[0249] Filtering the ox head region images based on quality detection of the ox head region images extracted from the initial images to obtain target ox head images;

[0250] Aligning the target ox head images to obtain aligned images, and performing instance segmentation on the front face images to obtain instance images;

[0251] Performing augmentation processing on the aligned images to obtain first augmented images, and performing augmentation processing on the instance images to obtain second augmented images;

[0252] Fusing face features of the first augmented images and local features of the second augmented images to obtain target features;

[0253] Performing object recognition based on the target features to obtain an object identity of the collection object.

[0254] The modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0255] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0256] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims involved.

[0257] Furthermore, the term "comprising" does not exclude other elements or steps, and the singular does not exclude the plural and vice-versa, unless the context clearly requires these exclusions. The disclosure of a single item, includes the disclosure of multiple items unless the context clearly dictates otherwise. The terms "comprising", "comprise" and "when comprising" specify the presence of stated features but do not preclude the presence or addition of one or more other features.

[0258] Finally, it should be noted that the above-mentioned embodiments illustrate rather than limit the application, since the scope of the application is determined by the appended claims.

Claims

1. A method for recognizing cow faces, characterized in that, The cow face recognition method includes: Acquire facial images of the subject from different orientations; Based on key point detection and quality detection of the facial image, the facial image is filtered to obtain an initial image, which includes a frontal facial image of the acquired object; Based on the quality detection of the bull head region image obtained from the semantic segmentation of the initial image, the bull head region image is filtered to obtain the target bull head image; Align the target bull head image to obtain an aligned image, and perform instance segmentation on the frontal face image to obtain instance images; The aligned image is subjected to normalized augmentation processing to obtain a first augmented image, and the instance image is subjected to normalized augmentation processing to obtain a second augmented image; The method of fusing facial features from the first augmented image and local features from the second augmented image to obtain target features includes: extracting facial features from the first augmented image based on a pre-trained first feature extraction model, wherein the facial features include facial feature elements at each matrix position; extracting local features from the second augmented image based on a pre-trained second feature extraction model, wherein the local features include local feature elements at each matrix position; performing a voting process on the facial feature elements and corresponding local feature elements at each matrix position to obtain a target element corresponding to each matrix position, wherein the target element is the element with the most values ​​among the facial feature elements and the corresponding local feature elements; and concatenating the target element based on multiple matrix positions to obtain the target features. Based on the target features, object identification is performed to obtain the object identity of the collected object.

2. The cow face recognition method as described in claim 1, characterized in that, The step of filtering the face image based on key point detection and quality detection to obtain the initial image includes: Key point detection is performed on the facial image to obtain facial key points; Identify the object category corresponding to the collected object; Based on the object configuration shape of the object category and the facial key points, a detection region is identified from the facial image; Calculate the pixel variance value of the detection area; Based on the comparison between the pixel variance value and the preset deviation threshold, the face image is filtered to obtain the filtered face image; Calculate the image sharpness of the filtered facial image; The filtered facial image is processed based on the image clarity to obtain the initial image.

3. The cow face recognition method as described in claim 2, characterized in that, The step of performing instance segmentation on the frontal face image to obtain instance images includes: Target detection is performed on the frontal face image to obtain the coordinates of the object center in the frontal face image; Obtain the offset threshold for the object category; Based on the object center coordinates and the offset threshold, the object contour image is identified from the frontal face image; Based on the object's configured shape, the image shape of the object's outline image is adjusted to obtain the target outline image; The target contour image is masked to obtain the instance image.

4. The cow face recognition method as described in claim 1, characterized in that, The step of filtering the bull head region image obtained from semantic segmentation of the initial image to obtain the target bull head image includes: Based on the pixel value and pixel position of each pixel in the initial image, identify the pixel type to which each pixel in the initial image belongs; The region formed by pixels of a preset type is defined as the bull's head region image; Calculate the ratio of the number of pixels in the bull's head region image to the image area of ​​the bull's head region image; Based on the ratio information and the preset ratio threshold, the bull head region image is filtered, and the filtered bull head region image is determined as the target bull head image.

5. The cow face recognition method as described in claim 1, characterized in that, Aligning the target bull's head image to obtain the aligned image includes: Key point pairs were identified from multiple target bull head images; Based on the coordinate information of the key point pairs in the multiple target bull head images, the alignment transformation relationship of the key point pairs is calculated; Based on the alignment transformation relationship, the multiple target bull head images are aligned to obtain the aligned image.

6. The cow face recognition method as described in claim 1, characterized in that, Before extracting the facial features from the first augmented image based on a pre-trained first feature extraction model, the method further includes: Obtain the first training image and first labeled features of the first training object, and obtain the second training image and second labeled features of the second training object; Based on a pre-built feature extraction network, feature extraction is performed on the first training image to obtain predicted features; Based on the first feature distance between the predicted feature and the first labeled feature, and the second feature distance between the predicted feature and the second labeled feature, the parameters of the feature extraction network are adjusted until the first feature distance is minimized and the second feature distance is maximized, thus obtaining the first feature extraction model.

7. A cow face recognition device, characterized in that, The cow face recognition device includes: The acquisition unit is used to acquire facial images of the subject from different orientations. A filtering unit is used to filter the face image based on key point detection and quality detection to obtain an initial image, wherein the initial image includes a frontal face image of the acquired object; The filtering unit is further configured to filter the bull head region image based on the quality detection of the bull head region image obtained from the semantic segmentation of the initial image, so as to obtain the target bull head image; The alignment segmentation unit is used to align the target bull head image to obtain an aligned image, and to perform instance segmentation on the frontal face image to obtain an instance image; An augmentation unit is used to perform normalized augmentation processing on the aligned image to obtain a first augmented image, and to perform normalized augmentation processing on the instance image to obtain a second augmented image; A fusion unit, used to fuse facial features of the first augmented image and local features of the second augmented image to obtain target features, includes: extracting facial features from the first augmented image based on a pre-trained first feature extraction model, wherein the facial features include facial feature elements at each matrix position; extracting local features from the second augmented image based on a pre-trained second feature extraction model, wherein the local features include local feature elements at each matrix position; performing a voting process on the facial feature elements and corresponding local feature elements at each matrix position to obtain a target element corresponding to each matrix position, wherein the target element is the element with the most values ​​among the facial feature elements and the corresponding local feature elements; and concatenating the target element based on multiple matrix positions to obtain the target features. The identification unit is used to identify objects based on the target features to obtain the object identity of the collected object.

8. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the cow face recognition method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the cow face recognition method as described in any one of claims 1 to 6.

Images