Chicken mouth opening target detection confidence improvement method and device based on double-model reasoning and medium

Abstract

The invention discloses a chicken mouth target detection confidence improvement method and device based on double-model reasoning and a medium, and the method comprises the steps: receiving a collected to-be-detected image, inputting the to-be-detected image into a reasoning model I for processing, carrying out the recognition and classification of the input image through the model, and carrying out the image classification through the confidence obtained through calculation in the recognition process; setting a first confidence coefficient and a second confidence coefficient as a classification basis; the images with the confidence degree between the first confidence degree and the second confidence degree are second-class images and are identified again through the inference model II; determining a reserved image and a confidence coefficient corresponding to the image; and outputting the image with the confidence higher than the third confidence for subsequent action analysis and judgment. The method has the advantages that the first model conducts efficient image rough recognition, the analysis quantity and difficulty are reduced, the second model selects the image part related to the content or behavior to be analyzed in the image with the medium confidence coefficient to conduct accurate recognition and analysis, the analysis difficulty and complexity are reduced, and the balance of accurate analysis and rapid analysis is achieved.

Description

Technical Field

[0001] The invention relates to a method, a device and a medium for improving the confidence of chicken open mouth target detection based on dual-model reasoning, and belongs to the technical field of breeding individual feature detection. Background Art

[0002] At present, in the caged broiler farming industry, the detection of broiler mouth-opening behavior is an urgent problem to be solved. First of all, broiler mouth-opening breathing may be a manifestation of physical discomfort. When broilers have difficulty breathing due to illness, they will open their mouths to breathe. Chickens that breathe with their mouths open are prone to respiratory diseases because open-mouthed breathing allows a large amount of dry air to enter the lungs directly, damaging the trachea and bronchial mucosa. Secondly, mouth-opening behavior can reflect the comfort of the breeding environment. For example, when the breeding temperature is too high or the ventilation is poor, broilers are prone to open-mouthed breathing. Finally, mouth-opening behavior will also affect the production performance of broilers, because open-mouthed breathing will affect the normal feeding and appetite of broilers, thereby affecting the feed intake.

[0003] In summary, the detection of broiler mouth-opening behavior is of great significance, but the detection of broiler mouth-opening behavior has problems such as small targets, complex backgrounds, and many targets to be detected, which makes it difficult to improve the precision and accuracy of chicken mouth-opening detection. The goal of the present invention is to design a method that can improve the confidence of broiler mouth-opening target detection, thereby laying a foundation for other subsequent studies based on broiler mouth-opening behavior. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, an object of the present invention is to provide a method for improving target detection confidence based on dual-model reasoning.

[0005] According to an embodiment of the present invention, a first solution is provided: A method for improving the confidence of chicken open beak target detection based on dual model reasoning includes the following steps: Receive the collected image to be tested and pre-process the image; The input is processed by the inference model 1, which recognizes and classifies the input image, aiming to detect as many chicken mouth targets as possible; The recognition and classification is to perform block recognition on the input image to be tested, identify the image part with the chicken's mouth-opening behavior characteristics in the image, calculate the confidence of the identified part, and classify the image according to the confidence calculated in the recognition process; Setting a first confidence level and a second confidence level as a basis for classification; Images with a confidence level higher than or equal to the first confidence level are classified as first-class images and are directly output for subsequent action analysis and judgment; Images with confidence levels lower than or equal to the second confidence level are classified as Class III images and are discarded for further analysis; The images with confidence between the first confidence and the second confidence are second-class images and will be recognized again through the second inference model; The second type of image is input into the second inference model for further image recognition. The model amplifies the second type of image, increases the pixel points of the image, and increases the analysis information of the model, in order to improve the accuracy of the recognition analysis and increase the confidence of the image used for subsequent analysis; After the two types of images are identified by the second inference model, the confidence level of the retained images and the corresponding images is determined; An image with a confidence level higher than the third confidence level is output for subsequent action analysis and judgment.

[0006] Furthermore, the second confidence level is 0.3, and images below the confidence level are considered to be elements and images irrelevant to the analysis, and these images are interference factors that need to be eliminated; The first confidence level is 0.8. The credibility of the image behavior action with a higher confidence level is extremely high, and it is directly output for subsequent recognition and analysis; Images with confidence levels between the first confidence level and the second confidence level need to be recognized and classified again. The secondary recognition inference model 2 is optimized and designed to improve its recognition and classification capabilities, thereby improving the confidence levels of the recognized and classified images.

[0007] Further, the third confidence level is 0.7; The steps to determine the retained image and the confidence level of the image correspondence are, Compare the confidence level A of the image before amplification and the confidence level B of the image after amplification; If the confidence B of the image after amplification is higher than the confidence A before amplification, the amplified image and the confidence B are marked and stored; If the confidence remains unchanged after amplification, the image and confidence A before amplification are retained; If the confidence B after amplification is lower than the confidence A before amplification, then determine whether the confidences of the two calculations are both greater than the third confidence; If yes, keep the original image and confidence A; If they are not both greater than the third confidence level, then compare whether the difference in confidence levels calculated by the two models is greater than a preset difference. If so, discard the images of the two models. If not, retain the image with the higher confidence level and the corresponding image. The default difference is 0.2; If both are less than the third confidence level, the images of the two models are discarded; The number of pixels of the added image is 300, and the directions of the image enlargement are four directions: up, down, left, and right.

[0008] Furthermore, the establishment of S2 reasoning model 1, Select yolov8 as the basic network, adjust the training parameters, and train the inference model 1. The specific implementation method is: S21, yolov8 is selected as the basic network. Compared with other yolo series algorithms, the main feature of this algorithm is that the C2f module is used instead of the C3 module, and the SPPF module is used instead of the SPP module. In the Head part, the original coupling head is also changed into a decoupling head; S22, set the training parameters of the model, including epoch, batch-size, image-size, The model training in S22 turned off mosaic enhancement in the last ten rounds of training, and the optimizer was selected as the settings including SGD; S23, use the training data of model 1 for training.

[0009] Furthermore, the establishment of S3 inference model 2, Train the second inference model. This model introduces the SPD-Conv module between the ordinary convolutions, and also adds the SEAttention mechanism and the BIFPN bidirectional weighted pyramid splicing module to play the interaction between the three modules, so that it can better identify the chicken's open mouth target; SPD-Conv, BIFPN, and SEAttention are combined and applied sequentially in the network improvement of Model 2. SPD-Conv is first used to efficiently extract spatial features of broiler mouth opening behavior; BIFPN is used to fuse the broiler mouth opening features extracted by SPD-Conv to enhance the expression of multi-scale information; SEAttention is placed after BIFPN, and the fused broiler mouth opening feature map is weighted at the channel level to improve the expression of important features and weaken irrelevant information; the specific steps are as follows: Add the selected combination modules to the basic network, namely, SPD-Conv, BIFPN, and SEAttention combination, and introduce the small object detection layer as the training network of model 2; Set the training parameters of the model, including epoch, batch-size, and image-size; Use the training data of model 2 for training.

[0010] Furthermore, the method for obtaining the training data of the model 1 is: S1, prepare the data for model training. Before establishing the inference model, you need to prepare the data set used for model training. The specific establishment method is as follows; SS1, collects and annotates images, annotates them through tools, and establishes a detection dataset for target images; SS11, image acquisition is performed by an image acquisition device, such as a camera, to monitor and obtain a low-light image of the detection target; SS12, use relevant tools to annotate and preprocess the collected images; SS13 performs data cleaning and preprocessing on the annotated images to establish a data set for detecting the mouth-opening behavior of caged broiler chickens; In the process of labeling data, the relevant tool used is labelImg tool; The information annotated in SS12 includes: the coordinate information of the mouth state when the broiler opens its mouth, the size information of the mouth, and the type of the mouth. The annotated information is stored in XML format. The preprocessing after annotation refers to converting all XML format files into TXT format to normalize the coordinates. The completed TXT format file contains the target category, the coordinates of the upper left corner, and the coordinates of the lower right corner. The data cleaning operation in SS13 screens out the poor quality data caused by manual operation problems; SS2, divides the data obtained by SS13 into two parts, and the first part is directly used for training model 1 without any operation.

[0011] Furthermore, the method for obtaining the training data of the second model is: SS2, the data obtained by SS13 is divided into two parts. The first part is used directly for training model 1 without any operation. The second part reads the annotation file and extracts the label position. Then the image is enlarged and cropped in four directions. The cropped image data is used for training model 2. The specific implementation method of SS2 is: SS21, the collected and labeled data are divided into two equal parts. No other operations are performed on the first part of the data, and it is directly divided into training set, validation set and test set in the ratio of 8:1:1, which is directly used as the training data of model 1; SS22, for the second part of the data, use Python program to read its XML annotation file to obtain the specific position and size of each annotation box, and expand it in four directions: up, down, left, and right. Each direction is expanded by 300 units of pixels and then cropped. After cropping, re-annotate the second part of the data after re-annotation. Use a 9:1 ratio to divide it into training set and validation set data as the training set and validation set data of model 2; SS23, the test set of model 2 is obtained as follows: Obtain the second category of images output after being identified and classified by the inference model 1 in the test set of the first part; The prediction boxes in the second category images are expanded in four directions: up, down, left, and right. The expanded pixels in each direction are 300 units and then cropped to obtain the enlarged second category image set, which is used as the test set data for model 2.

[0012] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: Receive the collected image to be tested and pre-process the image; The input is processed by the inference model 1, which recognizes and classifies the input image, aiming to detect as many chicken mouth targets as possible; The recognition and classification is to perform block recognition on the input image to be tested, identify the image part with the chicken's mouth-opening behavior characteristics in the image, calculate the confidence of the identified part, and classify the image according to the confidence calculated in the recognition process; Setting a first confidence level and a second confidence level as a basis for classification; Images with a confidence level higher than or equal to the first confidence level are classified as first-class images and are directly output for subsequent action analysis and judgment; Images with confidence levels lower than or equal to the second confidence level are classified as Class III images and are discarded for further analysis; The images with confidence between the first confidence and the second confidence are second-class images and will be recognized again through the second inference model; The second type of image is input into the second inference model for further image recognition. The model amplifies the second type of image, increases the pixel points of the image, and increases the analysis information of the model, in order to improve the accuracy of the recognition analysis and increase the confidence of the image used for subsequent analysis; After the two types of images are identified by the second inference model, the confidence level of the retained images and the corresponding images is determined; An image with a confidence level higher than the third confidence level is output for subsequent action analysis and judgment.

[0013] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following steps: Receive the collected image to be tested and pre-process the image; The input is processed by the inference model 1, which recognizes and classifies the input image, aiming to detect as many chicken mouth targets as possible; The recognition and classification is to perform block recognition on the input image to be tested, identify the image part with the chicken's mouth-opening behavior characteristics in the image, calculate the confidence of the identified part, and classify the image according to the confidence calculated in the recognition process; Setting a first confidence level and a second confidence level as a basis for classification; Images with a confidence level higher than or equal to the first confidence level are classified as first-class images and are directly output for subsequent action analysis and judgment; Images with confidence levels lower than or equal to the second confidence level are classified as Class III images and are discarded for further analysis; The images with confidence between the first confidence and the second confidence are second-class images and will be recognized again through the second inference model; The second type of image is input into the second inference model for further image recognition. The model amplifies the second type of image, increases the pixel points of the image, and increases the analysis information of the model, in order to improve the accuracy of the recognition analysis and increase the confidence of the image used for subsequent analysis; After the two types of images are identified by the second inference model, the confidence level of the retained images and the corresponding images is determined; An image with a confidence level higher than the third confidence level is output for subsequent action analysis and judgment.

[0014] Compared with the prior art, the technical solution provided by this application has unique beneficial effects.

[0015] The beneficial effect is that the first model performs efficient image preprocessing and classification, cutting, and rough recognition, which greatly reduces the number and difficulty of image analysis, extracts useful parts of the image, and performs rough confidence classification.

[0016] Through the second model, the image parts related to the content or behavior to be analyzed in the medium-confidence image are selected for accurate identification and analysis, which greatly reduces the difficulty and complexity of subsequent analysis and achieves a balance between accurate analysis and fast analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] in: Figure 1 It is a flow chart of the chicken open mouth target detection confidence improvement method based on dual model reasoning of the present invention; Figure 2 It is a schematic diagram of the overall process of the dual model construction and training process of the present invention; Figure 3 This is a schematic diagram of the network structure of the second inference model of the present invention; Figure 4Schematic diagram of the structure of the small target detection layer in the second inference model of the present invention; Figure 5 Schematic diagram of the SPD-Conv convolution module of the present invention; Figure 6 Schematic diagram of the BIFPN multi-scale feature fusion module of the present invention; Figure 7 This is a schematic diagram of the SEAttention mechanism module of the present invention; Figure 8 It is a structural block diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of this application.

[0020] Embodiment 1 A method for improving the confidence of chicken open mouth target detection based on dual model reasoning, the steps include: Receive the collected image to be tested and pre-process the image; The input is processed by the inference model 1, which recognizes and classifies the input image, aiming to detect as many chicken mouth targets as possible; The recognition and classification is to perform block recognition on the input image to be tested, identify the image part with the chicken's mouth-opening behavior characteristics in the image, calculate the confidence of the identified part, and classify the image according to the confidence calculated in the recognition process; Setting a first confidence level and a second confidence level as a basis for classification; Images with a confidence level higher than or equal to the first confidence level are classified as first-class images and are directly output for subsequent action analysis and judgment; Images with confidence levels lower than or equal to the second confidence level are classified as Class III images and are discarded for further analysis; Images with confidence levels between the first confidence level and the second confidence level are second-category images and will be re-identified using the second inference model.

[0021] The second confidence level is 0.3. Images with a confidence level lower than this level are considered to be elements and images irrelevant to the analysis, and these images are interference factors that need to be eliminated. The first confidence level is 0.8. The credibility of the image behavior action with a higher confidence level is extremely high, and it is directly output for subsequent recognition and analysis; For images with confidence levels between the first and second confidence levels, since the design of model one is based on the purpose of speed and efficiency, the accuracy is sacrificed. Therefore, this part of the image needs to be recognized and classified again. Since most of the images in this part already have characteristic pixels of some behavioral actions, it is necessary to optimize the design of the secondary recognition inference model two to improve its recognition and classification capabilities, so as to improve the confidence of the recognized and classified images.

[0022] S2, inputting the second type of image into the second inference model for further image recognition, the model amplifies the second type of image, increases the pixel points of the image, and increases the analysis information of the model, with the purpose of improving the accuracy of the recognition analysis and improving the confidence of the image used for subsequent analysis; After the two types of images are identified by the second inference model, the confidence level of the retained images and the corresponding images is determined; An image with a confidence level higher than the third confidence level is output for subsequent action analysis and judgment.

[0023] The third confidence level is 0.7.

[0024] The steps to determine the retained image and the confidence level of the image correspondence are, Compare the confidence level A of the image before amplification and the confidence level B of the image after amplification; If the confidence B of the image after the augmentation is higher than the confidence A before the augmentation, the augmented image and the confidence B are marked and stored; If the confidence remains unchanged after amplification, the image and confidence A before amplification are retained; If the confidence B after amplification is lower than the confidence A before amplification, then determine whether the confidences of the two calculations are both greater than the third confidence; If yes, keep the original image and confidence A; If they are not both greater than the third confidence level, then compare whether the difference in confidence levels calculated by the two models is greater than a preset difference. If so, discard the images of the two models. If not, retain the image with the higher confidence level and the corresponding image. The default difference is 0.2.

[0025] If both are less than the third confidence level, the images of the two models are discarded.

[0026] The number of pixels of the added image is 300, and the directions of the image enlargement are four directions: up, down, left, and right.

[0027] S1 prepares the data for model training. Before establishing the inference model, you need to prepare the data set used for model training. The specific establishment method is as follows.

[0028] The images collected and annotated by SS1 are annotated through tools to establish a detection dataset of the target image; SS11, image acquisition is performed by an image acquisition device, such as a camera, to monitor and obtain the low-light image of the detection target. SS12, use relevant tools to annotate and preprocess the collected images; SS13 performs data cleaning and preprocessing on the annotated images to establish a data set for detecting the mouth-opening behavior of caged broiler chickens.

[0029] It should be noted that in the process of labeling data, the relevant tool used is the labelImg tool.

[0030] The information annotated in SS12 includes: the coordinate information of the mouth state when the broiler opens its mouth, the size information of the mouth, and the type of the mouth. The annotated information is stored in XML format. The preprocessing after annotation refers to converting all XML format files into TXT format to normalize the coordinates. The completed TXT format file contains the target category, the coordinates of the upper left corner, and the coordinates of the lower right corner. The data cleaning operation in SS13 screens out poor-quality data caused by manual operation problems.

[0031] SS2, divides the data obtained by SS13 into two parts. The first part is used directly for training model one without any operation. The second part reads the annotation file and extracts the label position. Then the image is enlarged and cropped in four directions. The cropped image data is used for training model two.

[0032] The specific implementation method of SS2: SS21, the collected and labeled data are divided into two equal parts. No other operations are performed on the first part of the data, and it is directly divided into training set, validation set and test set in the ratio of 8:1:1, which is directly used as the training data of model 1; SS22, for the second part of the data, use Python program to read its XML annotation file to obtain the specific position and size of each annotation box, and expand it in four directions: up, down, left, and right. Each direction is expanded by 300 units of pixels and then cropped. After cropping, re-annotate the second part of the data after re-annotation. Use a 9:1 ratio to divide it into training set and validation set data as the training set and validation set data of model 2; SS23, the test set of model 2 is obtained as follows: Obtain the second category of images output after being identified and classified by the inference model 1 in the test set of the first part; The prediction boxes in the second category images are expanded in four directions: up, down, left, and right. The expanded pixels in each direction are 300 units and then cropped to obtain the enlarged second category image set, which is used as the test set data for model 2.

[0033] The establishment of S2 reasoning model 1, Select yolov8 as the basic network, adjust the training parameters, and train the inference model 1. The specific implementation method is: S21, yolov8 is selected as the basic network. Compared with other yolo series algorithms, the main feature of this algorithm is that the C2f module is used instead of the C3 module, and the SPPF module is used instead of the SPP module. In the Head part, the original coupling head is also changed into a decoupling head.

[0034] S22, set the training parameters of the model, including epoch, batch-size, image-size, The model training parameter settings in S22 include batch-size of 16, epoch of 250, initial learning rate of 0.001, image-size of 640×640, mosaic enhancement turned off in the last ten rounds of training, and SGD as the optimizer. These are all relatively good parameter settings after experimental verification.

[0035] S23, build a virtual environment for training on a 3090 server with a GPU memory of 24G, then input 80% of the training set of the broiler mouth-opening data set that has been divided for training inference model 1 into the model for detection training. After the training is completed, the inference model for broiler mouth-opening behavior detection is obtained, and then the verification set accounting for 10% of the data set is input into the inference model to verify the model. Finally, the test set accounting for 10% of the data set is input into the inference model to test the model, detect the model's time performance, storage size, recognition accuracy, etc., optimize the inference model, and finally obtain a good broiler mouth-opening behavior detection inference model.

[0036] The images with medium credibility are further analyzed and identified, and the images predicted to contain mouth-opening behavior are selected for subsequent analysis, and the images that are close to or interfere with mouth-opening behavior are excluded after further fine identification and analysis.

[0037] For example, the beak of a chicken is located on the head, which may be a very accurate identification location information, but due to the angle or position of the image shooting, the beak is captured and it is not possible to locate whether it is the head. Or the opening of a chicken's mouth is blocked by a certain part of another chicken's body, or the whole behavior combination is captured.

[0038] Another possibility is that the color of the chicken's feet is close to that of its beak, and some shooting angles may allow some toes to be close to the image, color, and shape of the open mouth.

[0039] All of the above situations need to be eliminated through further analysis. In the preliminary image analysis, a large amount of image information is not an image of a chicken opening its mouth. If one step is used for analysis, a lot of computing power will be wasted, and the accuracy of the information will also be affected. Therefore, a two-step method is used to improve the credibility of the final selected image, so that it can be better used for the final behavioral analysis and identification.

[0040] The beneficial effect is that the number of images to be analyzed and the difficulty of analysis are greatly reduced, and the image parts related to the content or behavior to be analyzed are selected for accurate identification and analysis, which greatly reduces the difficulty and complexity of subsequent analysis and achieves a balance between accurate analysis and fast analysis.

[0041] The establishment of S3 reasoning model 2, Train the second inference model. This model introduces the SPD-Conv module between the ordinary convolutions, and also adds the SEAttention mechanism and the BIFPN bidirectional weighted pyramid splicing module to play the interaction between the three modules, so that it can better identify the chicken's open mouth target; SPD-Conv, BIFPN, and SEAttention are used as a specific combination and in a specific order in the network improvement of Model 2. SPD-Conv is first used to efficiently extract spatial features of broiler mouth opening behavior; BIFPN is used to fuse the broiler mouth opening features extracted by SPD-Conv to enhance the expression of multi-scale information; SEAttention is placed after BIFPN, and channel-level attention weighting is performed on the fused broiler mouth opening feature map to enhance the expression of important features and weaken irrelevant information.

[0042] The specific role of each part in the combined module is as follows: SPD-Conv is placed in the early layers of the Backbone network to efficiently extract spatial features. It can replace the traditional convolution layer and efficiently capture low-level spatial information such as edges and textures in shallow networks. BIFPN is placed after Backbone and receives feature maps from different layers (usually feature maps of different resolutions). BiFPN enhances the expression of features by bidirectionally fusing these feature maps from different layers in the upstream and downstream directions. SEAttention is mainly used to weight the channel dimension and is placed after BIFPN. By weighting different channels, it can further enhance useful features and suppress irrelevant or noise information.

[0043] S31, add the selected combination module to the basic network, namely, SPD-Conv, BIFPN, and SEAttention combination, and introduce the small object detection layer as the training network of model two.

[0044] The size of the small target detection head added to the Head part of S31 is 128×128. The specific method is to add one more layer of upsampling to change the number of channels from 256 to 128, and then concatenate and reconvolve with the 128 layers of the Backbone part to output as a detection layer. The specific channel change is as follows: Figure 2 As shown in the figure; the SPD-Conv module is mainly added between the yolov8 convolution layer and C2f and between the convolution layer and the splicing layer. Specifically, six layers of SPD-Conv convolution layers are added between the first and second layers, the third and fourth layers, the fifth and sixth layers, the seventh and eighth layers, the sixteenth and seventeenth layers, and the nineteenth and twentieth layers; the opening behavior of broiler chickens may be disturbed by background noise in different environments, such as complex backgrounds or the movements of other chickens. The SPD-Conv module can filter out some noise by introducing frequency domain information and extract purer action features. At the same time, the chicken's mouth-opening action is subtle and fast. SPD-Conv can effectively extract the detailed features of the action at different scales, so as to make accurate judgments on short and inconspicuous mouth-opening actions. The BIFPN structure is added after the SPD-Conv convolution layer, mainly to replace the Concat structure. The opening behavior of broiler chickens presents different characteristics at different distances and angles. For example, the front picture lacks a lot of feature information compared with the side picture, and cannot fully show the mouth features of broiler chickens. BIFPN can effectively fuse feature maps of different scales, from fine-grained features to high-level semantic information, ensuring that even if the chicken's mouth opening action is very small, it can still be recognized. The SEAttention mechanism is added to the end of the network structure, after the BIFPN module, to further emphasize the feature information and increase the network's interest in the target area. The action of broiler chickens opening their mouths is usually accompanied by local deformations, such as the opening and closing of the mouth. In the image, these action features are often masked by features of other parts (such as feathers or body contours). SEAttention can automatically highlight important features related to the chicken's mouth by learning the weights of each channel, and enhance the model's attention to key areas. At the same time, in the broiler mouth recognition task, the background includes interference factors such as other chickens, feed or cages. SEAttention can improve the accuracy of recognition by suppressing the features of irrelevant or secondary channels and reducing the interference of background noise on recognition.

[0045] It should be noted that the difference between the SPD-Conv convolution layer and the ordinary convolution is that it downsamples the channel dimension of the feature map while retaining the information. This method can avoid the information loss in the traditional method. SPD-Conv is mainly composed of an SPD layer and a Non-strided convolution layer. SPD transforms the feature map. The main transformation is as follows:

[0046] Non-strided convolution is a The filter is a non-strided (i.e. stride=1) convolutional layer, where , so that the feature map is further transformed, the main transformation is as follows:

[0047] The reason for doing this is to retain all discriminant feature information as much as possible. The specific operation is as follows Figure 4 .

[0048] BIFPN refers to a bidirectional feature pyramid network. Its main features include efficient bidirectional cross-scale connections. It can be bidirectionally connected through two paths, top-down and bottom-up, allowing information from features of different scales to flow and fuse more effectively. It also has a relatively simplified network structure, and introduces learnable weights to determine the importance of different features, thereby improving the effect of feature fusion. Its weighting mainly refers to the fact that BIFPN adds an additional weight to each input and allows the network to learn the importance of each feature input, as shown in the following formula:

[0049] in is a learnable weight, which can be a scalar, vector, or multidimensional tensor. This weighted fusion method can achieve comparable accuracy to other methods while minimizing the computational cost. The specific structure is shown in the figure below. Figure 5 shown.

[0050] The added attention mechanism is SEAttention, which performs a global mean pooling operation on each channel to compress the channel into a scalar value. The scalar values ​​from multiple channels form a vector, which is processed by the fully connected layer and then transformed into values ​​between 0 and 1 using the sigmoid function. These values ​​correspond to the weights of different channels. The specific operation is as follows: Figure 6 .

[0051] S32, set the training parameters of the model, including epoch, batch-size, and image-size; The model training parameter settings in S32 include batch-size of 64, epoch of 300, initial learning rate of 0.001, image-size of 640×640, mosaic enhancement turned off in the last ten rounds of training, and SGD as the optimizer. These are relatively good parameter settings after experimental verification.

[0052] S33, build a virtual environment for training on the 3090 server with a GPU memory of 24G, and then input 90% of the training set of the broiler mouth-opening data set that has been divided for training the second inference model into the improved model for detection training. After the training is completed, the second inference model for broiler mouth-opening behavior detection is obtained, and then the verification set accounting for 10% of the data set is input into the second inference model to verify the model. Finally, the expanded second-category image set is used as the test set data for model two for evaluation and correction. Example

[0053] Perform the first recognition on the image to be detected and cut out the unreliable detection frame. The image to be detected is input into Model 1 for preliminary reasoning and rough recognition. Model 1 mainly does the following work: first, it identifies the location of the mouth-opening behavior and outputs the confidence result of the preliminary reasoning of the mouth-opening behavior recognition. Secondly, the detection frames with confidence lower than 0.3 are directly removed, and the detection frames with confidence higher than 0.8 are directly output as reliable results. The chicken mouth-opening detection frames with confidence between 0.3 and 0.8 are not reliable results and are left for the next step.

[0054] There are two main types of misjudgment in model 1. One is the open mouth situation with low confidence, and the other is a false high confidence. The reason for the misjudgment is that the size of the broiler open mouth image to be detected is 4032×3024, which loses a lot of image information when it is converted into a 640×640 image input for yolov8. At the same time, the background of the broiler open mouth image is complex and the detection target is small, which is also one of the important reasons for the misjudgment.

[0055] The mouth prediction boxes with confidence levels between 0.3 and 0.8 are enlarged in the four directions of up, down, left, and right. The enlarged size is 300 pixels each, and then the enlarged images are cropped.

[0056] The reason for enlarging by 300 pixels in all four directions is that the proportion of the enlarged broiler's mouth image in the overall image can reach 10%-40%, which is beneficial to the secondary reasoning below. If the mouth image accounts for too small a proportion of the overall image (accounting for < 10%), the detection model may have difficulty capturing enough features, especially when the resolution is low. In this case, target details will be lost, resulting in false detection or missed detection. If the mouth image accounts for too large a proportion of the overall image (accounting for > 40%), the detector cannot capture the complete contextual information, resulting in inaccurate positioning of the mouth, reducing the accuracy of model 2.

[0057] Input the amplification prediction boxes with confidence between 0.3 and 0.8 after trimming into model 2. Use model 2 for secondary reasoning to achieve confidence correction, thereby outputting reliable mouth opening recognition information; Use model 2 to correct the confidence of the detection boxes output by model 1 with a confidence level between 0.3 and 0.8, and perform secondary reasoning on the existing misjudgments (low confidence and false high confidence). Re-determine the mouth opening status and output a reliable mouth opening recognition result The reason why Model 2 can achieve confidence correction is that the Model 2 used in this step uses a dataset that reads label files and then expands and crops them since training. Compared with the original large-size images, this dataset is closer to the input size of the yolov8 network training, thereby reducing the information loss of the image.

[0058] Secondly, the SPD-Conv convolution used in model 2 can filter out some noise by introducing frequency domain information to extract purer mouth-opening action features in response to the background noise interference of the chicken's mouth-opening action in different environments. BIFPN can effectively fuse feature maps of different scales based on the subtlety of the chicken's mouth-opening action and the different features it presents at different distances and angles, ensuring accurate identification of the broiler's mouth-opening behavior. The SEAttention mechanism can automatically highlight the important features related to the chicken's mouth by learning the weights of each channel, targeting the easily concealed characteristics of the broiler's mouth-opening action, and enhancing the model's attention to key areas. Finally, BIFPN can fuse the features extracted by SPD-Conv, and SEAttention can weight the features fused by BIFPN. The combination of the three modules enables them to play a better role, thereby achieving the correction of confidence and improving the credibility of the detection results.

[0059] Output the mouth prediction box with high confidence after secondary inference and the overall average accuracy.

[0060] The output mouth opening prediction frame with high confidence is the confidence of the prediction frame with confidence between 0.3 and 0.8 of model 1 after correction, which is the secondary inference and screening of the prediction results with low credibility of model 1. It can ensure the accuracy of the output results.

[0061] Statistical test results, main indicators are P, R, F1, etc.

[0062] At the same time, the occurrence location and coordinate information of the broiler's mouth-opening behavior can also be found in the output file, realizing automatic recognition of the broiler's mouth-opening behavior.

[0063] The specific observation results are shown in the following table:

[0064] The original inference model uses yolov8 as the basic architecture. In terms of convolution, it mainly uses 3×3 and 1×1 convolution kernels. In terms of feature fusion, it uses the SPPF structure. In terms of detection, it mainly uses detection heads of three sizes: 256×256, 512×512, and 1024×1024.

[0065] From the above data table, we can see that compared with the original model, the dual model reasoning has better performance in detecting the opening mouth behavior of broiler chickens, specifically, P is improved by about 23.4%, R is improved by about 22.6%, mAP50 is improved by about 20.4%, and mAP50-95 is improved by about 66.2%. These data show that the average accuracy of the dual model reasoning at different IoU thresholds is better than that of the original model reasoning.

[0066] Embodiment 3 Figure 8 FIG. 1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a terminal or a server. Figure 8 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement a method for improving the confidence of chicken open mouth target detection based on dual model reasoning. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute a method for improving the confidence of chicken open mouth target detection based on dual model reasoning. Those skilled in the art can understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0067] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: Receive the collected image to be tested and pre-process the image; The input is processed by the inference model 1, which recognizes and classifies the input image, aiming to detect as many chicken mouth targets as possible; The recognition and classification is to perform block recognition on the input image to be tested, identify the image part with the chicken's mouth-opening behavior characteristics in the image, calculate the confidence of the identified part, and classify the image according to the confidence calculated in the recognition process; Setting a first confidence level and a second confidence level as a basis for classification; Images with a confidence level higher than or equal to the first confidence level are classified as first-class images and are directly output for subsequent action analysis and judgment; Images with confidence levels lower than or equal to the second confidence level are classified as Class III images and are discarded for further analysis; The images with confidence between the first confidence and the second confidence are second-class images and will be recognized again through the second inference model; The second type of image is input into the second inference model for further image recognition. The model amplifies the second type of image, increases the pixel points of the image, and increases the analysis information of the model, in order to improve the accuracy of the recognition analysis and increase the confidence of the image used for subsequent analysis; After the two types of images are identified by the second inference model, the confidence level of the retained images and the corresponding images is determined; An image with a confidence level higher than the third confidence level is output for subsequent action analysis and judgment.

[0068] In one embodiment, a computer-readable storage medium is provided, storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps: Receive the collected image to be tested and pre-process the image; The input is processed by the inference model 1, which recognizes and classifies the input image, aiming to detect as many chicken mouth targets as possible; The recognition and classification is to perform block recognition on the input image to be tested, identify the image part with the chicken's mouth-opening behavior characteristics in the image, calculate the confidence of the identified part, and classify the image according to the confidence calculated in the recognition process; Setting a first confidence level and a second confidence level as a basis for classification; Images with a confidence level higher than or equal to the first confidence level are classified as first-class images and are directly output for subsequent action analysis and judgment; Images with confidence levels lower than or equal to the second confidence level are classified as Class III images and are discarded for further analysis; The images with confidence between the first confidence and the second confidence are second-class images and will be recognized again through the second inference model; The second type of image is input into the second inference model for further image recognition. The model amplifies the second type of image, increases the pixel points of the image, and increases the analysis information of the model, in order to improve the accuracy of the recognition analysis and increase the confidence of the image used for subsequent analysis; After the two types of images are identified by the second inference model, the confidence level of the retained images and the corresponding images is determined; An image with a confidence level higher than the third confidence level is output for subsequent action analysis and judgment.

[0069] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0070] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for improving the confidence of chicken open mouth target detection based on dual model reasoning, characterized in that: The steps include: Receive the collected image to be tested and pre-process the image; The input is processed by the inference model 1, which recognizes and classifies the input image, aiming to detect as many chicken mouth targets as possible; The recognition and classification is to perform block recognition on the input image to be tested, identify the image part with the chicken's mouth-opening behavior characteristics in the image, calculate the confidence of the identified part, and classify the image according to the confidence calculated in the recognition process; Setting a first confidence level and a second confidence level as a basis for classification; Images with a confidence level higher than or equal to the first confidence level are classified as first-class images and are directly output for subsequent action analysis and judgment; Images with confidence levels lower than or equal to the second confidence level are classified as Class III images and are discarded for further analysis; The images with confidence between the first confidence and the second confidence are second-class images and will be recognized again through the second inference model; The second type of image is input into the second inference model for further image recognition. The model amplifies the second type of image, increases the pixel points of the image, and increases the analysis information of the model, in order to improve the accuracy of the recognition analysis and increase the confidence of the image used for subsequent analysis; After the two types of images are identified by the second inference model, the confidence level of the retained images and the corresponding images is determined; An image with a confidence level higher than the third confidence level is output for subsequent action analysis and judgment.

2. The method for improving the confidence of chicken open mouth target detection based on dual model reasoning according to claim 1 is characterized in that: include: The second confidence level is 0.

3. Images with a confidence level lower than this level are considered to be elements and images irrelevant to the analysis, and these images are interference factors that need to be eliminated. The first confidence level is 0.

8. The credibility of the image behavior action with a higher confidence level is extremely high, and it is directly output for subsequent recognition and analysis; Images with confidence levels between the first confidence level and the second confidence level need to be recognized and classified again. The secondary recognition inference model 2 is optimized and designed to improve its recognition and classification capabilities, thereby improving the confidence levels of the recognized and classified images.

3. The method for improving the confidence of chicken open mouth target detection based on dual model reasoning according to claim 2 is characterized in that: include: The third confidence level is 0.7; The steps to determine the retained image and the confidence level of the image correspondence are, Compare the confidence level A of the image before amplification and the confidence level B of the image after amplification; If the confidence B of the image after amplification is higher than the confidence A before amplification, the amplified image and the confidence B are marked and stored; If the confidence remains unchanged after amplification, the image and confidence A before amplification are retained; If the confidence B after amplification is lower than the confidence A before amplification, then determine whether the confidences of the two calculations are both greater than the third confidence; If yes, keep the original image and confidence A; If they are not both greater than the third confidence level, then compare whether the difference in confidence levels calculated by the two models is greater than a preset difference. If so, discard the images of the two models. If not, retain the image with the higher confidence level and the corresponding image. The default difference is 0.2; If both are less than the third confidence level, the images of the two models are discarded; The number of pixels of the added image is 300, and the directions of the image enlargement are four directions: up, down, left, and right.

4. The method for improving the confidence of chicken open mouth target detection based on dual model reasoning according to claim 3 is characterized in that: include: The establishment of S2 reasoning model 1, Select yolov8 as the basic network, adjust the training parameters, and train the inference model 1. The specific implementation method is: S21, yolov8 is selected as the basic network. Compared with other yolo series algorithms, the main feature of this algorithm is that the C2f module is used instead of the C3 module, and the SPPF module is used instead of the SPP module. In the Head part, the original coupling head is also changed into a decoupling head; S22, set the training parameters of the model, including epoch, batch-size, image-size, The model training in S22 turned off mosaic enhancement in the last ten rounds of training, and the optimizer was selected as the settings including SGD; S23, use the training data of model 1 for training.

5. The chicken open mouth target detection confidence enhancement method based on dual model reasoning according to claim 4 is characterized in that: include: The establishment of S3 reasoning model 2, Train the second inference model. This model introduces the SPD-Conv module between the ordinary convolutions, and also adds the SEAttention mechanism and the BIFPN bidirectional weighted pyramid splicing module to play the interaction between the three modules, so that it can better identify the chicken's mouth target; SPD-Conv, BIFPN, and SEAttention are combined and applied sequentially in the network improvement of Model 2. SPD-Conv is first used to efficiently extract spatial features of broiler mouth opening behavior; BIFPN is used to fuse the broiler mouth opening features extracted by SPD-Conv to enhance the expression of multi-scale information; SEAttention is placed after BIFPN, and the fused broiler mouth opening feature map is weighted at the channel level to improve the expression of important features and weaken irrelevant information; the specific steps are as follows: Add the selected combination modules to the basic network, namely, SPD-Conv, BIFPN, and SEAttention combination, and introduce the small object detection layer as the training network of model 2; Set the training parameters of the model, including epoch, batch-size, and image-size; Use the training data of model 2 for training.

6. The method for improving the confidence of chicken open mouth target detection based on dual model reasoning according to claim 5 is characterized in that: include: The method for obtaining the training data of the model 1 is: S1, prepare the data for model training. Before establishing the inference model, you need to prepare the data set used for model training. The specific establishment method is as follows; SS1, collects and annotates images, annotates them through tools, and establishes a detection dataset for target images; SS11, image acquisition is performed by an image acquisition device, such as a camera, to monitor and obtain a low-light image of the detection target; SS12, use relevant tools to annotate and preprocess the collected images; SS13 performs data cleaning and preprocessing on the annotated images to establish a data set for detecting the mouth-opening behavior of caged broiler chickens; In the process of labeling data, the relevant tool used is labelImg tool; The information annotated in SS12 includes: the coordinate information of the mouth state when the broiler opens its mouth, the size information of the mouth, and the type of the mouth. The annotated information is stored in XML format. The preprocessing after annotation refers to converting all XML format files into TXT format to normalize the coordinates. The completed TXT format file contains the target category, the coordinates of the upper left corner, and the coordinates of the lower right corner. The data cleaning operation in SS13 screens out the poor quality data caused by manual operation problems; SS2, divides the data obtained by SS13 into two parts, and the first part is directly used for training model 1 without any operation.

7. The method for improving the confidence of chicken open mouth target detection based on dual model reasoning according to claim 6 is characterized in that: include: The method for obtaining the training data of the model 2 is as follows: SS2, the data obtained by SS13 is divided into two parts. The first part is used directly for training model 1 without any operation. The second part reads the annotation file and extracts the label position. Then the image is enlarged and cropped in four directions. The cropped image data is used for training model 2. The specific implementation method of SS2 is: SS21, the collected and labeled data are divided into two equal parts. No other operations are performed on the first part of the data, and it is directly divided into training set, validation set and test set in the ratio of 8:1:1, which is directly used as the training data of model 1; SS22, for the second part of the data, use Python program to read its XML annotation file to obtain the specific position and size of each annotation box, and expand it in four directions: up, down, left, and right. Each direction is expanded by 300 units of pixels and then cropped. After cropping, re-annotate the second part of the data after re-annotation. Use a 9:1 ratio to divide it into training set and validation set data as the training set and validation set data of model 2; SS23, the test set of model 2 is obtained as follows: Obtain the second category of images output after being identified and classified by the inference model 1 in the test set of the first part; The prediction boxes in the second category images are expanded in four directions: up, down, left, and right. The expanded pixels in each direction are 300 units and then cropped to obtain the enlarged second category image set, which is used as the test set data for model 2.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

Images