Multi-pig instance segmentation method, device, equipment and medium

Abstract

The application relates to a multi-pig instance segmentation method, device, equipment and medium. The method comprises the following steps: channel splicing is performed on intermediate image features and a first segmentation mask to generate combined features; the combined features are input into a preset trainable adapter module; multi-scale analysis and fusion are performed on the combined features by the trainable adapter module to output a single-channel correction residual map; pixel-by-pixel addition operation is performed on the single-channel correction residual map and the first segmentation mask to generate a second segmentation mask for each pig instance; full-pixel traversal detection is performed on a second segmentation mask set to identify overlapping pixel points covered by the second segmentation masks corresponding to multiple pig instances; a pig instance corresponding to a highest prediction probability value is determined through maximum value operation; and each overlapping pixel point is assigned to the pig instance, and an instance segmentation result corresponding to the multiple pig instances is output. The application can cope with complex situations such as pig density in a pig farm and maintain stable and accurate segmentation effect.

Description

Technical Field

[0001] This application relates to the field of smart farming, and in particular to a method for segmenting multiple pig instances, a corresponding device, electronic equipment, and a computer-readable storage medium. Background Technology

[0002] Instance segmentation is a core technology in the field of smart farming. It can provide pixel-level accurate contours for each pig in a pig farm and is the technological cornerstone for realizing advanced downstream applications such as non-contact weight estimation, feeding behavior analysis, individual health monitoring and disease early warning.

[0003] However, deploying high-precision instance segmentation systems in real-world, large-scale pig farm environments faces significant challenges. Traditional instance segmentation models, such as Mask R-CNN, typically require training from scratch using labeled data from a specific pig farm. These models suffer from severe "domain drift," meaning that a model trained on one pig farm (e.g., with specific lighting, pen design, and pig breeds) experiences a sharp decline in segmentation performance when directly applied to another pig farm with a different environment. To maintain high accuracy, it is necessary to re-label and retrain the model for each new pig farm—a costly and time-consuming process that significantly hinders the large-scale deployment of the technology.

[0004] In recent years, general-purpose visual foundation models, such as the Segment Anything Model (SAM), have demonstrated powerful generalization capabilities, providing new ideas for solving the aforementioned problems. These models utilize massive amounts of pre-trained data and can segment objects in various scenarios without requiring specific training. However, when directly applied to the specific field of smart pig farms, new problems arise:

[0005] First, the lack of domain-specific knowledge (a lack of prior knowledge about pig morphology) in general vision-based basic models leads to insufficient segmentation accuracy. When pigs occlude each other or are obscured by feeding troughs or fences, the general model tends to segment only the visible parts (modal segmentation), rather than inferring and completing the occluded outlines based on the pig's body features (non-modal segmentation) like an experienced farmer. This can cause serious biases in downstream applications such as behavioral analysis.

[0006] Secondly, it cannot handle logical conflicts between instances. In a pigpen, pigs often cluster tightly together, their bodies overlapping. General models typically generate segmentation masks independently for each detected pig, inevitably leading to multiple masks simultaneously claiming ownership of the same pixel area in overlapping regions, producing logically and physically incorrect and confusing results.

[0007] In theory, fully fine-tuning the entire massive base model could inject domain knowledge into it, alleviating the aforementioned problems. However, fully fine-tuning a base model with hundreds of millions or even billions of parameters requires massive amounts of labeled data and extremely powerful GPU computing power, which is impractical for most aquaculture companies in terms of both hardware and time costs.

[0008] In summary, to address the problems in existing technologies regarding segmentation of multiple pig instances, such as the poor versatility of traditional small models and their inability to be deployed on a large scale, while powerful general-purpose visual foundation models are difficult to implement due to excessively high fine-tuning costs, the applicant has made corresponding explorations to solve these problems. Summary of the Invention

[0009] The purpose of this application is to solve the above-mentioned problems by providing a method, apparatus, electronic device and computer-readable storage medium for segmenting multiple pig instances.

[0010] To achieve the various objectives of this application, the following technical solution is adopted:

[0011] A method for segmenting multiple pig instances, proposed to suit one of the purposes of this application, includes:

[0012] Obtain monitoring images of the pig farm to be detected, which contain multiple pig instances;

[0013] The monitoring image of the pig farm to be detected is input into the encoder of a pre-trained visual large model with frozen weights. After forward propagation processing by the encoder, the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance, are output.

[0014] The intermediate image features and the first segmentation mask are concatenated by channels to generate combined features. The combined features are then input into a preset trainable adapter module, which performs multi-scale analysis and fusion on the combined features to output a single-channel corrected residual map. The single-channel corrected residual map represents the pixel-level difference between the first segmentation mask and the true contour of the pig instance.

[0015] The single-channel corrected residual image is added pixel by pixel to the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances.

[0016] A full pixel traversal detection is performed on the second segmentation mask set to identify overlapping pixels covered by the second segmentation mask corresponding to multiple pig instances. The predicted probability value of each pig instance at the corresponding overlapping pixel is retrieved. The pig instance corresponding to the highest predicted probability value is determined by taking the maximum value. Each overlapping pixel is assigned to the pig instance. After the assignment of all overlapping pixels is determined, the instance segmentation result corresponding to multiple pig instances is output.

[0017] Optionally, the step of inputting the monitoring image of the pig farm to be detected into the encoder of a pre-trained large visual model with frozen weights, performing forward propagation processing through the encoder, and outputting the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance, includes:

[0018] After standardizing the monitoring images of the pig farm to be detected, they are input into the encoder of the pre-trained large visual model.

[0019] In the encoder, the standardized pig farm monitoring image to be detected is sequentially processed by block embedding, multi-head self-attention calculation, and forward propagation of feedforward neural network mapping, and outputs intermediate image features that characterize the visual features of each pig instance.

[0020] A first segmentation mask is generated for each pig instance based on the intermediate image features, wherein the channel dimension of the intermediate image features matches the channel dimension of the first segmentation mask.

[0021] Optionally, the step of concatenating the intermediate image features and the first segmentation mask into channels to generate combined features, inputting the combined features into a preset trainable adapter module, and having the trainable adapter module perform multi-scale analysis and fusion on the combined features to output a single-channel corrected residual map includes:

[0022] Obtain the intermediate image features and the first segmentation mask corresponding to each pig instance, perform a channel concatenation operation on the intermediate image features and the first segmentation mask, and concatenate and fuse the channel dimensions of the intermediate image features and the channel dimensions of the first segmentation mask to generate combined features;

[0023] The combined features are input into a preset trainable adapter module, and multi-scale downsampling feature extraction, skip connection feature fusion, and upsampling feature restoration are sequentially performed on the combined features. The combined features at different scales are fused and enhanced layer by layer, and a single-channel corrected residual map with the same size as the first segmentation mask is output.

[0024] Optionally, the step of performing pixel-by-pixel addition of the single-channel corrected residual image and the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances, includes:

[0025] Obtain a single-channel corrected residual image and a first segmentation mask that correspond one-to-one with each pig instance in the monitoring image of the pig farm to be detected;

[0026] A pixel-by-pixel addition operation is performed on the corrected residual map and the corresponding first segmentation mask. The contour deviation of the first segmentation mask is compensated by pixel-level residual correction, and a second segmentation mask with complete contour and high fidelity is generated for each pig instance.

[0027] The second segmentation mask of all pig instances in the monitoring image of the pig farm to be detected is collected to form a second segmentation mask set that matches the number of pig instances.

[0028] Optionally, the steps of performing full pixel traversal detection on the second segmentation mask set to identify overlapping pixels covered by the second segmentation mask corresponding to multiple pig instances, retrieving the predicted probability value of each pig instance at the corresponding overlapping pixel, determining the pig instance corresponding to the highest predicted probability value by taking the maximum value, and assigning each overlapping pixel to the pig instance include:

[0029] Obtain the second segmentation mask set corresponding to all pig instances in the pig farm monitoring image to be detected, perform a full pixel traversal detection operation on the second segmentation mask set, and determine pixel by pixel whether each pixel in the pig farm monitoring image to be detected is simultaneously covered by the second segmentation mask corresponding to multiple pig instances in the second segmentation mask set, so as to filter and identify all overlapping pixel points.

[0030] Retrieve the mask prediction probability value corresponding to each pig instance at each of the overlapping pixels, and perform a maximum value operation on all prediction probability values ​​of the same overlapping pixel to determine the pig instance corresponding to the pixel attribution result.

[0031] Each overlapping pixel is assigned a unique ownership to the corresponding pig instance, thus completing the ownership determination of all overlapping pixels and ensuring that the second segmentation mask corresponding to each pig instance has no pixel overlap.

[0032] Optionally, the underlying network architecture of the visual large model includes the ViT architecture.

[0033] Optionally, the underlying network architecture of the trainable adapter module includes a lightweight convolutional neural network.

[0034] A multi-pig instance segmentation device provided for another purpose of this application includes:

[0035] The pig farm image acquisition module is configured to acquire monitoring images of a pig farm to be detected that contain multiple pig instances;

[0036] The first mask determination module is configured to input the monitoring image of the pig farm to be detected into the encoder of a pre-trained visual large model with frozen weights, perform forward propagation processing through the encoder, and output the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance.

[0037] The modified residual map determination module is configured to perform channel concatenation on the intermediate image features and the first segmentation mask to generate combined features, and input the combined features to a preset trainable adapter module. The trainable adapter module performs multi-scale analysis and fusion on the combined features to output a single-channel modified residual map, wherein the single-channel modified residual map represents the pixel-level difference between the first segmentation mask and the true contour of the pig instance.

[0038] The second mask determination module is configured to perform pixel-by-pixel addition of the single-channel corrected residual image and the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances.

[0039] The pig instance segmentation module is configured to perform full pixel traversal detection on the second segmentation mask set, identify overlapping pixels covered by the second segmentation mask corresponding to multiple pig instances, retrieve the predicted probability value of each pig instance at the corresponding overlapping pixel, determine the pig instance corresponding to the highest predicted probability value by taking the maximum value, and assign each overlapping pixel to the pig instance. After completing the assignment determination of all overlapping pixels, the module outputs the instance segmentation results corresponding to multiple pig instances.

[0040] An electronic device provided for another purpose of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the multi-pig instance segmentation method of this application.

[0041] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the multi-pig instance segmentation method, which, when invoked by a computer, performs the steps included in the corresponding method.

[0042] Compared to existing technologies, this application addresses the problems in existing technologies regarding segmentation of multiple pig instances, such as the poor versatility of traditional small models, which cannot be deployed on a large scale, and the difficulty in implementing powerful general-purpose vision-based large models due to excessively high fine-tuning costs. This application includes, but is not limited to, the following beneficial effects:

[0043] Firstly, by keeping the weights of the pre-trained visual large model backbone frozen and training only the lightweight adapter module, we can fully reuse the general visual capabilities of the large model, significantly reduce training computation and time costs, avoid retraining the backbone of the large model, and ensure segmentation accuracy and engineering deployment efficiency.

[0044] Secondly, by fusing intermediate image features with the first segmentation mask through channel splicing, the adapter learns semantic features and initial contour information simultaneously, outputting a single-channel corrected residual map to achieve pixel-level contour completion and deviation correction of the initial mask, effectively solving problems such as pig occlusion and rough boundaries, and generating a high-fidelity second segmentation mask.

[0045] Thirdly, for dense pig herd scenarios, overlapping pixels are located by full pixel traversal detection. The maximum value operation is performed based on the model's predicted probability to uniquely assign conflicting pixels to the pig instance with the highest confidence. This method eliminates mask overlap simply and efficiently, outputting unambiguous and conflict-free instance segmentation results, thus avoiding the logical conflict problem between instances that is difficult for neural networks to solve.

[0046] Furthermore, this application can handle complex situations in pig farms, such as densely packed pigs, mutual occlusion, and varied postures, maintaining stable and accurate segmentation results even under non-ideal monitoring conditions, meeting the actual monitoring needs of large-scale farming. The design of the lightweight adapter module and the overlapping pixel post-processing module significantly improves the algorithm's operating efficiency while ensuring segmentation accuracy, adapting to the real-time processing requirements of pig farm monitoring images, and facilitating local deployment and edge computing applications. The non-modal annotation training and residual correction mechanism makes the pig mask contours more complete; the probability-guided attribution determination rule makes the pixel allocation logic interpretable and traceable, providing a reliable data foundation for subsequent pig counting, behavior analysis, and other farming management applications. Attached Figure Description

[0047] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0048] Figure 1 This is a flowchart illustrating the method for segmenting multiple pig instances in an embodiment of this application.

[0049] Figure 2 This is a flowchart illustrating the workflow of the multi-pig instance segmentation system in this application embodiment;

[0050] Figure 3This is a schematic diagram comparing modal segmentation and non-modal segmentation under occlusion conditions in an embodiment of this application;

[0051] Figure 4 This is a schematic diagram illustrating pixel repetition conflict in a dense pig herd scenario as described in this application embodiment;

[0052] Figure 5 This is a schematic diagram illustrating the pixel attribution determination performed by the overlapping pixel post-processing module in an embodiment of this application.

[0053] Figure 6 This is a schematic diagram of the multi-pig instance segmentation device in the embodiments of this application;

[0054] Figure 7 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0055] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0056] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0057] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0058] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDAs (Personal Digital Assistants) that may include radio frequency receivers, pagers, internet / intranet access, web browsers, notebooks, calendars, and / or GPS (Global Positioning System) receivers; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0059] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0060] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0061] One or more of the technical features of this application, unless explicitly specified herein, can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or can be directly deployed and run on a client for access.

[0062] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0063] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0064] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0065] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0066] Please see Figure 1 as well as Figure 2 In one embodiment of the multi-pig instance segmentation method of this application, the method includes:

[0067] Step S10: Obtain a monitoring image of the pig farm to be detected, which contains multiple pig instances;

[0068] The multi-pig instance segmentation system in the terminal device can acquire monitoring images of a pig farm to be detected that contain multiple pig instances.

[0069] In some embodiments, the core idea of ​​this application is to train only a lightweight trainable adapter module to learn domain knowledge for pig segmentation while maintaining the backbone weights of a powerful pre-trained basic visual model. A deterministic probability guides the post-processing steps to correct inter-instance logical conflicts that the model itself struggles to learn. Specifically, the multi-pig instance segmentation method of this application includes several core stages, including:

[0070] Firstly, a pre-trained large-scale visual model encoder based on the ViT (Vision Transformer) architecture with frozen weights is used to perform tensor normalization on the input pig farm monitoring images. Then, forward propagation processing, including block embedding, multi-head self-attention calculation, and feedforward neural network mapping, is performed to generate a high-dimensional intermediate image feature tensor containing rich semantic information for each detected pig instance, as well as a preliminary low-resolution first segmentation mask tensor that may have problems such as inaccurate or incomplete contours. The channel dimensions of the intermediate image feature tensor and the first segmentation mask tensor are matched to provide a data foundation for subsequent feature fusion.

[0071] Secondly, this application designs a trainable adapter module based on a lightweight convolutional neural network (CNN), with a parameter count far smaller than that of the backbone model. The input to this trainable adapter module is a feature fusion operation via channel concatenation, combining the intermediate image feature tensor obtained in the previous step with the first segmentation mask tensor to generate a combined feature tensor. The trainable adapter module does not directly predict the final mask; instead, it is trained to learn a single-channel correction residual image. This residual image represents "what pixel-level adjustments are needed to the initial mask to transform it into a realistic pig outline," guiding the pixel-by-pixel addition and subtraction adjustments to the initial mask. By adding the single-channel correction residual image pixel-by-pixel to the first segmentation mask, a refined, more complete, and realistically fitting high-fidelity second segmentation mask tensor is obtained, and these are aggregated to form a second segmentation mask set. To enable the model to learn to complete the outline of occluded pigs, the adapter's training data preferably uses non-modal annotation.

[0072] Thirdly, it is extremely difficult for neural networks to perfectly resolve instance overlap conflicts in dense pig populations during training. Therefore, this application separates this problem from the training phase and designs a simple yet efficient probability-guided deterministic post-processing module that runs during inference. Its working principle is as follows: The system first generates a second segmentation mask for each pig instance in the graph, forming a set of second segmentation masks. Then, it performs a full pixel traversal detection on the second segmentation mask set. If a pixel is simultaneously covered by the masks of two or more pigs, the module retrieves the predicted probability values ​​of those pigs' masks at that pixel, calculates the maximum value, determines the pig instance with the highest probability value, and assigns the final ownership of that pixel to that pig instance. This "winner-takes-all" rule can simply and efficiently eliminate pixel overlap between all instances, ensuring that the second segmentation masks corresponding to each pig have no pixel overlap and no ownership conflicts, ultimately outputting a complete multi-pig instance segmentation result.

[0073] In some embodiments, please refer to Figure 3 , Figure 3 This invention illustrates the difference between modal segmentation and non-modal segmentation annotation methods when pigs are occluded in a pig farm scenario. Modal segmentation (red mask) only annotates the visible area, excluding the occluded portion (blue horizontal bar area), resulting in an incomplete pig outline. Non-modal segmentation (blue mask) annotates the complete pig outline, completing the pig body mask even when some areas are occluded, thus better reflecting the real pig's morphology. This application preferably uses non-modal annotation data to train an adapter module, enabling it to learn to complete the outline of occluded pigs, thereby correcting the incomplete outline problem of the first segmentation mask (initial mask) and improving the integrity and fidelity of the second segmentation mask.

[0074] In some embodiments, please refer to Figure 4 , Figure 4 This is used to illustrate the core problem of multi-instance segmentation in dense pig herd scenarios. When multiple pigs are close to each other and their limbs overlap, the coarse segmentation mask will assign the pixels of the overlapping area to multiple pig instances simultaneously. Figure 4 The yellow highlighted area represents a pixel-classification conflict region, leading to mask overlap and blurred boundaries between instances, making it impossible to distinguish individual pigs. This problem is difficult to solve directly through neural network training. Therefore, this application designs a probability-guided overlapping pixel post-processing module to remove and process such overlapping conflicts during the inference stage, ensuring the instance independence of the final segmentation result.

[0075] Step S20: Input the monitoring image of the pig farm to be detected into the encoder of the pre-trained visual large model with frozen weights. After forward propagation processing by the encoder, output the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, and the first segmentation mask of each pig instance.

[0076] After acquiring a monitoring image of a pig farm containing multiple pig instances, the monitoring image of the pig farm to be detected is input into the encoder of a pre-trained visual large model with frozen weights. After forward propagation processing by the encoder, the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance, are output. The basic network architecture of the visual large model includes the ViT architecture.

[0077] In some embodiments, the step of inputting the pig farm monitoring image to be detected into the encoder of a pre-trained large visual model with frozen weights, performing forward propagation processing through the encoder, and outputting the intermediate image features corresponding to each pig instance in the pig farm monitoring image to be detected, and the first segmentation mask of each pig instance, includes:

[0078] Step S201: After standardizing the monitoring image of the pig farm to be detected, input it into the encoder of the pre-trained visual large model;

[0079] Step S202: In the encoder, the standardized pig farm monitoring image to be detected is sequentially processed by block embedding, multi-head self-attention calculation, and forward propagation of feedforward neural network mapping, and the intermediate image features representing the visual features of each pig instance are output.

[0080] Step S203: Generate a first segmentation mask corresponding to each pig instance based on the intermediate image features, wherein the channel dimension of the intermediate image features matches the channel dimension of the first segmentation mask.

[0081] As can be seen from steps S201 to S203, standardizing the pig farm monitoring images before inputting them into the encoder eliminates the interference of environmental factors such as lighting, pixel scale, and color deviation in the actual pig farm scene. This ensures that the input data of the visual large model encoder conforms to the pre-trained feature distribution, avoids feature extraction deviation caused by differences in input data, and improves the stability and accuracy of feature and mask generation.

[0082] The forward propagation is performed by following the steps of block embedding, multi-head self-attention calculation, and feedforward neural network mapping. It is precisely adapted to the core working logic of the visual large model encoder based on the ViT architecture. It can effectively extract high-dimensional and semantically rich intermediate image features of each pig instance in the image to be detected, give full play to the general feature extraction capability of the pre-trained visual large model, and lay a high-quality feature foundation for subsequent mask generation and residual correction.

[0083] The first segmentation mask for the corresponding pig instance is directly generated based on the extracted intermediate image features, ensuring that the channel dimensions of the two are matched without the need for additional dimension transformation. This achieves integrated connection between feature extraction and initial mask generation, and provides direct dimensional compatibility for subsequent channel splicing operations of intermediate image features and the first segmentation mask. This simplifies the subsequent feature fusion process and improves the overall algorithm efficiency.

[0084] For each pig instance, corresponding intermediate image features and a first segmentation mask are generated independently, realizing a one-to-one correspondence between pig instances and features and masks. This provides accurate initial instantiation basis for subsequent steps such as mask refinement for single pigs and determination of overlapping pixel attribution, which meets the task requirements of multi-pig instance segmentation.

[0085] Step S30: Channel splicing is performed on the intermediate image features and the first segmentation mask to generate combined features. The combined features are input to a preset trainable adapter module, which performs multi-scale analysis and fusion on the combined features to output a single-channel corrected residual map. The single-channel corrected residual map represents the pixel-level difference between the first segmentation mask and the real contour of the pig instance.

[0086] The monitoring image of the pig farm to be detected is input into the encoder of a pre-trained visual large model with frozen weights. After forward propagation processing by the encoder, the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance, are output. Then, the intermediate image features and the first segmentation mask are concatenated by channel to generate combined features. The combined features are input into a preset trainable adapter module, which performs multi-scale analysis and fusion on the combined features to output a single-channel corrected residual map. The single-channel corrected residual map represents the pixel-level difference between the first segmentation mask and the true contour of the pig instance. The basic network architecture of the trainable adapter module includes a lightweight convolutional neural network.

[0087] In some embodiments, the step of concatenating the intermediate image features and the first segmentation mask into channels to generate combined features, inputting the combined features into a preset trainable adapter module, and having the trainable adapter module perform multi-scale analysis and fusion on the combined features to output a single-channel corrected residual map includes:

[0088] Step S301: Obtain the intermediate image features and the first segmentation mask corresponding to each pig instance, and perform a channel splicing operation on the intermediate image features and the first segmentation mask to splice and fuse the channel dimension of the intermediate image features with the channel dimension of the first segmentation mask to generate combined features;

[0089] Step S302: Input the combined features into a preset trainable adapter module, and sequentially perform multi-scale downsampling feature extraction, skip connection feature fusion, and upsampling feature restoration on the combined features. Perform layer-by-layer fusion and feature enhancement on the combined features of different scales, and output a single-channel corrected residual map with the same size as the first segmentation mask.

[0090] As can be seen from steps S301 to S302, by channel splicing, the intermediate image features corresponding to the pig instance are dimensionally fused with the first segmentation mask to generate combined features. This not only preserves the high-dimensional semantic information of the intermediate image features, but also incorporates the pixel-level contour information of the first segmentation mask. This allows the adapter module to learn the residual correction rules based on both semantic features and initial contour features, laying a complete feature foundation for accurately generating the corrected residual map.

[0091] By sequentially performing multi-scale downsampling extraction, skip connection fusion, and upsampling restoration on the combined features, it can capture the contour features (such as edge details and overall body shape) of pig instances at different scales. At the same time, skip connections are used to compensate for the feature loss of downsampling, realizing the layer-by-layer fusion and enhancement of features at different scales, which is suitable for complex segmentation scenarios in large-scale pig farms where pigs have different postures and occlusions.

[0092] The adapter module outputs a single-channel corrected residual map that is exactly the same size as the first segmentation mask. No additional preprocessing operations such as size transformation and channel dimensionality reduction are required. It can be directly added pixel by pixel with the first segmentation mask, which simplifies the subsequent mask refinement process and improves the overall running efficiency of the algorithm. At the same time, the single-channel design accurately matches the needs of the mask pixel-level correction, allowing the residual map to directly guide the contour deviation compensation of the initial mask.

[0093] The adapter module learns the contour residual correction rules of pig instances based only on the fused combined features, without having to relearn the complete segmentation mask generation logic. The number of parameters is much smaller than that of the main body of the large vision model, which greatly reduces the training computing power requirements and time costs of the model. At the same time, the lightweight model structure is more suitable for the local deployment needs of large-scale pig farms, balancing segmentation accuracy and engineering practicality.

[0094] The generated single-channel corrected residual map represents the pixel-level difference between the first segmentation mask and the real outline of the pig instance. It can provide accurate pixel-level adjustment guidance for the subsequent outline completion and deviation correction of the first segmentation mask, effectively solving the problems of rough outline and incomplete occlusion area of ​​the first segmentation mask generated by the pre-trained large visual model, and improving the fidelity of the pig instance mask.

[0095] Step S40: Add the single-channel corrected residual image to the first segmentation mask pixel by pixel to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances.

[0096] The monitoring image of the pig farm to be detected is input into the encoder of a pre-trained visual large model with frozen weights. After forward propagation processing by the encoder, the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance, are output. Then, the single-channel corrected residual image is added to the first segmentation mask pixel by pixel to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the monitoring image of the pig farm to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances.

[0097] In some embodiments, the step of performing pixel-by-pixel addition of the single-channel corrected residual image and the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances, includes:

[0098] Step S401: Obtain a single-channel corrected residual image and a first segmentation mask that correspond one-to-one with each pig instance in the monitoring image of the pig farm to be detected;

[0099] Step S402: Perform a pixel-by-pixel addition operation on the corrected residual map and the corresponding first segmentation mask, and compensate for the contour deviation of the first segmentation mask through pixel-level residual correction, so as to generate a second segmentation mask with complete contour and high fidelity for each pig instance.

[0100] Step S403: Collect the second segmentation masks of all pig instances in the monitoring image of the pig farm to be detected, and construct a second segmentation mask set that matches the number of pig instances.

[0101] As can be seen from steps S401 to S403, the single-channel corrected residual map and the first segmentation mask corresponding to each pig instance are accurately obtained, realizing the one-to-one binding of data and pig instances. This allows the residual correction operation to be performed independently for the initial mask contour deviation of a single pig, avoiding confusion between the features and masks of different pig instances, and ensuring the accuracy and relevance of mask refinement.

[0102] By performing residual correction through pixel-by-pixel addition, the first segmentation mask is directly compensated for pixel-level contour deviation based on the single-channel correction residual map. This can accurately complete the contour of the occluded area of ​​the pig instance, correct the rough mask boundary, efficiently solve the problem of incomplete contour and low fidelity of the first segmentation mask, and quickly generate a second segmentation mask with complete contour and close to the real body shape of the pig.

[0103] The single-channel corrected residual map is perfectly matched with the size of the first segmentation mask, and pixel-by-pixel addition can be performed directly without additional preprocessing such as size transformation and channel adaptation. This simplifies the mask refinement process, reduces the computational load of the algorithm, and effectively improves the mask refinement efficiency for individual pigs and the entire pig herd.

[0104] The second segmentation mask of all pig instances is aggregated into a second segmentation mask set that matches the number of instances. This achieves a systematic integration of the refined masks of multiple pig instances. It not only clearly distinguishes the independent masks of each pig instance, but also provides a structured data foundation for subsequent post-processing operations such as full pixel traversal of the mask set and determination of overlapping pixel attribution.

[0105] The refined second segmentation mask combines complete contour shape with high pixel fidelity, effectively making up for the segmentation defects of the initial mask of the pre-trained large visual model. It lays a high-quality mask foundation for resolving mask overlap conflicts and outputting unambiguous final segmentation results, thereby improving the overall accuracy of pig instance segmentation.

[0106] Step S50: Perform full pixel traversal detection on the second segmentation mask set, identify overlapping pixels covered by the second segmentation mask corresponding to multiple pig instances, retrieve the predicted probability value of each pig instance at the corresponding overlapping pixel, determine the pig instance corresponding to the highest predicted probability value by taking the maximum value, and assign each overlapping pixel to the pig instance. After completing the determination of the ownership of all overlapping pixels, output the instance segmentation result corresponding to multiple pig instances.

[0107] The single-channel corrected residual image is added pixel-by-pixel to the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected. After constructing a set of second segmentation masks corresponding to multiple pig instances, a full pixel traversal detection is performed on the second segmentation mask set to identify overlapping pixels covered by the second segmentation masks corresponding to multiple pig instances. The predicted probability value of each pig instance at the corresponding overlapping pixel is retrieved, and the pig instance corresponding to the highest predicted probability value is determined by taking the maximum value. Each overlapping pixel is assigned to the pig instance. After completing the assignment determination of all overlapping pixels, the instance segmentation result corresponding to multiple pig instances is output.

[0108] In some embodiments, the steps of performing full pixel traversal detection on the second segmentation mask set, identifying overlapping pixels covered by the second segmentation masks corresponding to multiple pig instances, retrieving the predicted probability value of each pig instance at the corresponding overlapping pixel, determining the pig instance corresponding to the highest predicted probability value by taking the maximum value, and assigning each overlapping pixel to the pig instance include:

[0109] Step S501: Obtain the second segmentation mask set corresponding to all pig instances in the pig farm monitoring image to be detected, perform a full pixel traversal detection operation on the second segmentation mask set, and determine pixel by pixel whether each pixel in the pig farm monitoring image to be detected is simultaneously covered by the second segmentation mask corresponding to multiple pig instances in the second segmentation mask set, so as to filter and identify all overlapping pixel points.

[0110] Step S502: Retrieve the mask prediction probability value corresponding to each pig instance at each of the overlapping pixels, and perform the maximum value operation on all prediction probability values ​​of the same overlapping pixel to determine the pig instance corresponding to the pixel attribution result.

[0111] Step S503: Assign the unique ownership of each overlapping pixel to the corresponding pig instance, and complete the ownership determination of all overlapping pixels so that the second segmentation mask corresponding to each pig instance has no pixel overlap.

[0112] As can be seen from steps S501 to S503, full pixel traversal detection is performed on the second segmentation mask set. Only overlapping pixels that are simultaneously covered by multiple pig instance masks are selected and identified, rather than traversing the entire image. This not only accurately locates the core conflict area of ​​multi-pig instance segmentation, but also avoids invalid processing of non-conflicting background / single instance pixels, thus improving post-processing efficiency.

[0113] Based on the mask prediction probability values ​​of overlapping pixels for each pig instance, the attribution is determined by taking the maximum value. The confidence of the model prediction is the core basis, rather than subjective rules, so that the pixel attribution results are more in line with the real contour features of the pig instance. This effectively avoids the segmentation bias caused by manual rule allocation and improves the accuracy of multi-instance segmentation.

[0114] By assigning a unique pig instance ownership to each overlapping pixel, the pixel overlap problem between multiple pig instance masks is directly eliminated, so that the second segmentation mask of each pig instance forms an independent contour without conflict or ambiguity, thus meeting the core requirement of "instance independence and no pixel overlap" for multi-target instance segmentation.

[0115] The entire attribution determination process is non-randomized, requiring no additional training or parameter adjustment. While resolving overlapping conflicts, it avoids introducing new computational complexity, balancing segmentation accuracy and algorithm efficiency, and is suitable for the engineering needs of real-time segmentation of pig farm monitoring images.

[0116] The entire process of traversing, judging, and allocating the second segmentation mask set is carried out, always anchoring the independent attribute of "pig instance". It not only preserves the integrity of each instance mask, but also achieves systematic optimization of the mask set through pixel assignment adjustment, perfectly adapting to the instance segmentation needs of large-scale pig farms with dense multi-pig scenarios.

[0117] In a specific embodiment, the multi-pig instance segmentation system described in this application mainly includes frozen weights, a large visual model based on the ViT architecture, a trainable adapter module based on a lightweight convolutional neural network (CNN), and an overlapping pixel post-processing module.

[0118] When the system is working, firstly, the input pig farm monitoring image to be detected can be preprocessed by a lightweight object detection model to generate bounding boxes as target cues for all pig instances in the image. Subsequently, these target cues, along with the pig farm monitoring image, are input into the encoder of a ViT-based visual large model with frozen weights. For each pig target cue provided by an external detector or interactive operation in the pig farm monitoring image, the image is first standardized and then input into the encoder of the ViT-based visual large model. The encoder sequentially performs block embedding, multi-head self-attention calculation, and forward propagation processing of feedforward neural network mapping on the standardized image, outputting high-dimensional intermediate image features and a low-resolution first segmentation mask. The intermediate image features represent the visual semantic features of each pig instance. The first segmentation mask (initial segmentation mask) may have problems such as rough boundaries and incomplete contours. The channel dimension of the intermediate image features matches the channel dimension of the first segmentation mask, providing a data foundation for subsequent feature fusion.

[0119] Furthermore, the intermediate image features and the first segmentation mask are fed together into a trainable adapter module based on a lightweight convolutional neural network (CNN). The lightweight convolutional neural network (CNN) can be a U-shaped or U-like structure, etc. The feature fusion operation at the input end adopts channel concatenation, which concatenates the 256-channel intermediate image features with the 1-channel first segmentation mask to form a 257-channel combined feature tensor.

[0120] In the trainable adapter module, through its encoder-decoder structure and skip connections, the combined features are processed layer by layer through multi-scale downsampling feature extraction, skip connection feature fusion, and upsampling feature restoration, and finally output a single-channel corrected residual map. This single-channel corrected residual map represents the pixel-level difference between the first segmentation mask and the real contour of the pig instance, and is used to guide the contour compensation of the initial mask.

[0121] Furthermore, a mask refiner performs a pixel-by-pixel addition operation on the single-channel corrected residual map and the corresponding first segmentation mask, and compensates for the contour deviation of the initial mask through pixel-level residual correction, generating a second segmentation mask with complete contour and high fidelity for each pig instance.

[0122] Repeat the above process independently for each pig instance in the monitoring image of the pig farm to be detected, thereby obtaining a set of second segmentation masks. ,in, The number of pigs is represented by the second segmentation mask. The second segmentation mask for all pig instances is aggregated to form a set of second segmentation masks that matches the number of pigs.

[0123] The total mask loss function used by the adapter module during training The expression is represented as:

[0124]

[0125] in, This represents the total mask loss function used during adapter module training, which measures the difference between the predicted mask and the real pig outline, and guides model parameter updates. FocalLoss focuses on optimizing the prediction accuracy of hard-to-classify pixels (such as pig edges, occluded areas, etc.) to alleviate the problem of imbalanced samples. DiceLoss, which focuses on optimizing the similarity between the overall shape of the mask and the real contour, improves the continuity of region segmentation; Indicated for balance The hyperparameter weights control the proportion of focus loss in the total loss. Indicated for balance The hyperparameter weights control the proportion of the dice loss in the total loss. Non-modal labeled data is preferred during adapter training to improve the ability to complete the contours of occluded pigs.

[0126] During the inference phase, the second segmentation mask set is fed into the probability-guided overlapping pixel post-processing module to eliminate pixel overlap conflicts between pig instances. The module performs full pixel traversal detection on the second segmentation mask set, determining pixel by pixel whether the point is simultaneously covered by the second segmentation mask corresponding to multiple pigs, and filtering and identifying all overlapping pixel points.

[0127] Specifically, for any pixel in the monitoring image of the pig farm to be detected, the second segmentation mask set covers... The overlapping pixel post-processing module first determines whether the pixel is simultaneously covered by the second segmentation mask of multiple pig instances in the second segmentation mask set; if so, it retrieves all pig instances covering that pixel. Predicted probabilities on their respective masks .

[0128] Then, for all predicted probability values ​​of the same overlapping pixel, the maximum value (argmax) is calculated, and the final ownership of that overlapping pixel is uniquely assigned to the pig instance with the highest probability value. After determining the ownership of all overlapping pixels, the system outputs a set of completely conflict-free final instance segmentation masks for each pixel. The final ownership determination rule, and its calculation formula, are as follows:

[0129]

[0130] in, Represents pixels The final assignment result represents the unique pig instance to which the pixel belongs; This represents the pixel coordinates in the monitoring image of the pig farm to be inspected. The horizontal axis is... The vertical axis; Indicates the number of covered pixels The set of all pig instances, i.e., which pigs' second segmentation masks simultaneously cover the pixel; Represents a set The index of a single pig instance in the table represents a specific pig instance; Represents a pig instance The refined second segmentation mask at the pixel point The predicted probability value represents the model's assessment that the pixel belongs to a pig instance. The probability of; This represents the maximum value operation (argmax operation) in a set. Find Largest index Thus determining the pixel point Examples of pigs to which the species belongs.

[0131] In some embodiments, please refer to Figure 5 , Figure 5 This section explains the core logic of the overlapping pixel post-processing module in resolving pixel overlap conflicts. During the coarse masking stage, overlapping pixel blocks are simultaneously assigned to both "pig a" (blue) and "pig b" (red), resulting in an assignment conflict. The overlapping pixel post-processing module retrieves the values ​​of "pig a" and "pig b" at the overlapping pixel location. The predicted probability value , By taking the maximum value operation This pixel is uniquely assigned to the pig instance with the higher probability value. Figure 5 (The data is classified as pig a). Based on the probability determination rule, the mask overlap between multiple pig instances can be eliminated simply and efficiently, and the final instance segmentation result is output without conflict or ambiguity.

[0132] As can be seen from the above embodiments, compared with the prior art, this application addresses the problems in the prior art regarding the segmentation of multiple pig instances, such as the poor versatility of traditional small models and their inability to be deployed on a large scale, while the large, powerful general-purpose vision model is difficult to implement due to the high cost of fine-tuning. This application has, but is not limited to, the following beneficial effects:

[0133] Firstly, by keeping the weights of the pre-trained visual large model backbone frozen and training only the lightweight adapter module, we can fully reuse the general visual capabilities of the large model, significantly reduce training computation and time costs, avoid retraining the backbone of the large model, and ensure segmentation accuracy and engineering deployment efficiency.

[0134] Secondly, by fusing intermediate image features with the first segmentation mask through channel splicing, the adapter learns semantic features and initial contour information simultaneously, outputting a single-channel corrected residual map to achieve pixel-level contour completion and deviation correction of the initial mask, effectively solving problems such as pig occlusion and rough boundaries, and generating a high-fidelity second segmentation mask.

[0135] Thirdly, for dense pig herd scenarios, overlapping pixels are located by full pixel traversal detection. The maximum value operation is performed based on the model's predicted probability to uniquely assign conflicting pixels to the pig instance with the highest confidence. This method eliminates mask overlap simply and efficiently, outputting unambiguous and conflict-free instance segmentation results, thus avoiding the logical conflict problem between instances that is difficult for neural networks to solve.

[0136] Furthermore, this application can handle complex situations in pig farms, such as densely packed pigs, mutual occlusion, and varied postures, maintaining stable and accurate segmentation results even under non-ideal monitoring conditions, meeting the actual monitoring needs of large-scale farming. The design of the lightweight adapter module and the overlapping pixel post-processing module significantly improves the algorithm's operating efficiency while ensuring segmentation accuracy, adapting to the real-time processing requirements of pig farm monitoring images, and facilitating local deployment and edge computing applications. The non-modal annotation training and residual correction mechanism makes the pig mask contours more complete; the probability-guided attribution determination rule makes the pixel allocation logic interpretable and traceable, providing a reliable data foundation for subsequent pig counting, behavior analysis, and other farming management applications.

[0137] Please see Figure 6 A multi-pig instance segmentation device provided for one of the purposes of this application includes a pig farm image acquisition module 1100, a first mask determination module 1200, a corrected residual image determination module 1300, a second mask determination module 1400, and a pig instance segmentation module 1500. The system includes a pig farm image acquisition module 1100, configured to acquire a pig farm monitoring image containing multiple pig instances; a first mask determination module 1200, configured to input the pig farm monitoring image to be detected into the encoder of a pre-trained visual large model with frozen weights, perform forward propagation processing through the encoder, and output the intermediate image features corresponding to each pig instance in the pig farm monitoring image to be detected, as well as the first segmentation mask for each pig instance; and a corrected residual map determination module 1300, configured to perform channel concatenation of the intermediate image features and the first segmentation mask to generate combined features, input the combined features into a preset trainable adapter module, and perform multi-scale analysis and fusion of the combined features through the trainable adapter module to output a single-channel corrected residual map, wherein the single-channel corrected residual map represents the first segmentation mask and the first segmentation mask. The true outline pixel-level differences of the pig instances; the second mask determination module 1400 is configured to perform pixel-by-pixel addition operation on the single-channel corrected residual map and the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances; the pig instance segmentation module 1500 is configured to perform full pixel traversal detection on the second segmentation mask set, identify overlapping pixels covered by the second segmentation masks corresponding to multiple pig instances, retrieve the predicted probability value of each pig instance at the corresponding overlapping pixel, determine the pig instance corresponding to the highest predicted probability value by taking the maximum value operation, and assign each overlapping pixel to the pig instance, and after completing the assignment determination of all overlapping pixels, output the instance segmentation result corresponding to multiple pig instances.

[0138] Based on any embodiment of this application, please refer to Figure 7Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as... Figure 7 The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, they enable the processor to implement a multi-pig instance segmentation method. The processor of the computer device provides computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store computer-readable instructions, which, when executed by the processor, enable the processor to execute the multi-pig instance segmentation method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0139] In this embodiment, the processor is used to execute... Figure 6 The specific functions of each module are described, and the memory stores the program code and various data required to execute the above modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules in the multi-pig instance segmentation device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0140] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the multi-pig instance segmentation method described in any embodiment of this application.

[0141] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the multi-pig instance segmentation method described in any embodiment of this application.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0143] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for segmenting multiple pig instances, characterized in that, include: Obtain monitoring images of the pig farm to be detected, which contain multiple pig instances; The monitoring image of the pig farm to be detected is input into the encoder of a pre-trained visual large model with frozen weights. After forward propagation processing by the encoder, the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance, are output. The intermediate image features and the first segmentation mask are concatenated by channels to generate combined features. The combined features are then input into a preset trainable adapter module, which performs multi-scale analysis and fusion on the combined features to output a single-channel corrected residual map. The single-channel corrected residual map represents the pixel-level difference between the first segmentation mask and the true contour of the pig instance. The single-channel corrected residual image is added pixel by pixel to the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances. A full pixel traversal detection is performed on the second segmentation mask set to identify overlapping pixels covered by the second segmentation mask corresponding to multiple pig instances. The predicted probability value of each pig instance at the corresponding overlapping pixel is retrieved. The pig instance corresponding to the highest predicted probability value is determined by taking the maximum value. Each overlapping pixel is assigned to the pig instance. After the assignment of all overlapping pixels is determined, the instance segmentation result corresponding to multiple pig instances is output.

2. The method for segmenting multiple pig instances according to claim 1, characterized in that, The steps of inputting the monitoring image of the pig farm to be detected into the encoder of a pre-trained large visual model with frozen weights, performing forward propagation processing through the encoder, and outputting the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance, include: After standardizing the monitoring images of the pig farm to be detected, they are input into the encoder of the pre-trained large visual model. In the encoder, the standardized pig farm monitoring image to be detected is sequentially processed by block embedding, multi-head self-attention calculation, and forward propagation of feedforward neural network mapping, and outputs intermediate image features that characterize the visual features of each pig instance. A first segmentation mask is generated for each pig instance based on the intermediate image features, wherein the channel dimension of the intermediate image features matches the channel dimension of the first segmentation mask.

3. The method for segmenting multiple pig instances according to claim 1, characterized in that, The steps of concatenating the intermediate image features and the first segmentation mask to generate combined features, inputting the combined features into a preset trainable adapter module, and having the trainable adapter module perform multi-scale analysis and fusion on the combined features to output a single-channel corrected residual map include: Obtain the intermediate image features and the first segmentation mask corresponding to each pig instance, perform a channel concatenation operation on the intermediate image features and the first segmentation mask, and concatenate and fuse the channel dimensions of the intermediate image features and the channel dimensions of the first segmentation mask to generate combined features; The combined features are input into a preset trainable adapter module, and multi-scale downsampling feature extraction, skip connection feature fusion, and upsampling feature restoration are sequentially performed on the combined features. The combined features at different scales are fused and enhanced layer by layer, and a single-channel corrected residual map with the same size as the first segmentation mask is output.

4. The method for segmenting multiple pig instances according to claim 1, characterized in that, The step of performing pixel-by-pixel addition between the single-channel corrected residual image and the first segmentation mask to generate a high-fidelity second segmentation mask with complete outlines for each pig instance in the pig farm monitoring image to be detected, and constructing a set of second segmentation masks corresponding to multiple pig instances, includes: Obtain a single-channel corrected residual image and a first segmentation mask that correspond one-to-one with each pig instance in the monitoring image of the pig farm to be detected; A pixel-by-pixel addition operation is performed on the corrected residual map and the corresponding first segmentation mask. The contour deviation of the first segmentation mask is compensated by pixel-level residual correction, and a second segmentation mask with complete contour and high fidelity is generated for each pig instance. The second segmentation mask of all pig instances in the monitoring image of the pig farm to be detected is collected to form a second segmentation mask set that matches the number of pig instances.

5. The method for segmenting multiple pig instances according to claim 1, characterized in that, The steps include performing a full pixel traversal detection on the second segmentation mask set, identifying overlapping pixels covered by the second segmentation masks corresponding to multiple pig instances, retrieving the predicted probability value of each pig instance at the corresponding overlapping pixels, determining the pig instance corresponding to the highest predicted probability value by taking the maximum value, and assigning each overlapping pixel to the pig instance. Obtain the second segmentation mask set corresponding to all pig instances in the pig farm monitoring image to be detected, perform a full pixel traversal detection operation on the second segmentation mask set, and determine pixel by pixel whether each pixel in the pig farm monitoring image to be detected is simultaneously covered by the second segmentation mask corresponding to multiple pig instances in the second segmentation mask set, so as to filter and identify all overlapping pixel points. Retrieve the mask prediction probability value corresponding to each pig instance at each of the overlapping pixels, and perform a maximum value operation on all prediction probability values ​​of the same overlapping pixel to determine the pig instance corresponding to the pixel attribution result. Each overlapping pixel is assigned a unique ownership to the corresponding pig instance, thus completing the ownership determination of all overlapping pixels and ensuring that the second segmentation mask corresponding to each pig instance has no pixel overlap.

6. The method for segmenting multiple pig instances according to any one of claims 1 to 5, characterized in that, The underlying network architecture of the large visual model includes the ViT architecture.

7. The method for segmenting multiple pig instances according to any one of claims 1 to 5, characterized in that, The basic network architecture of the trainable adapter module includes a lightweight convolutional neural network.

8. A multi-pig instance segmentation device, characterized in that, include: The pig farm image acquisition module is configured to acquire monitoring images of a pig farm to be detected that contain multiple pig instances; The first mask determination module is configured to input the monitoring image of the pig farm to be detected into the encoder of a pre-trained visual large model with frozen weights, perform forward propagation processing through the encoder, and output the intermediate image features corresponding to each pig instance in the monitoring image of the pig farm to be detected, as well as the first segmentation mask of each pig instance. The modified residual map determination module is configured to perform channel concatenation on the intermediate image features and the first segmentation mask to generate combined features, and input the combined features to a preset trainable adapter module. The trainable adapter module performs multi-scale analysis and fusion on the combined features to output a single-channel modified residual map, wherein the single-channel modified residual map represents the pixel-level difference between the first segmentation mask and the true contour of the pig instance. The second mask determination module is configured to perform pixel-by-pixel addition of the single-channel corrected residual image and the first segmentation mask to generate a second segmentation mask with complete outline and high fidelity for each pig instance in the pig farm monitoring image to be detected, so as to construct a set of second segmentation masks corresponding to multiple pig instances. The pig instance segmentation module is configured to perform full pixel traversal detection on the second segmentation mask set, identify overlapping pixels covered by the second segmentation mask corresponding to multiple pig instances, retrieve the predicted probability value of each pig instance at the corresponding overlapping pixel, determine the pig instance corresponding to the highest predicted probability value by taking the maximum value, and assign each overlapping pixel to the pig instance. After completing the assignment determination of all overlapping pixels, the module outputs the instance segmentation results corresponding to multiple pig instances.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images