Cow farm scene semantic segmentation method and device, equipment and medium

Abstract

The invention provides a cattle farm scene semantic segmentation method and device, equipment and a medium. The method comprises the steps of obtaining pictures of a road center, a near fence and an obstacle area in a cattle farm; the pictures are input into a cattle farm image semantic segmentation model established based on an improved SegFormer network architecture for semantic segmentation processing, cattle farm recognition is completed, and the cattle farm image semantic segmentation model comprises an encoder and a decoder; the encoder adopts a layered Transform structure, and a multi-scale cavity attention module is cascaded behind each Transform Block to form a dual feature enhancement unit; the decoder comprises a lightweight multi-layer perceptron layer, a hierarchical efficient feature fusion module and a classification head, and the lightweight multi-layer perceptron layer is used for carrying out channel dimension unification on a plurality of feature maps with different resolutions output by the multi-scale cavity attention module; and the hierarchical efficient feature fusion module is used for performing feature refining on the feature map with uniform channel dimensions, and the classification head performs category prediction on each pixel in the feature map after feature refining. The method can be widely applied to cattle farm scene recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and agricultural robots, and particularly to a beef field scene semantic segmentation method based on an improved SegFormer model, a device, equipment and a medium. BACKGROUND

[0002] At present, agricultural robots mainly realize autonomous navigation function through global navigation satellite system (GNSS), inertial navigation system (INS), light detection and ranging (LiDAR) and machine vision navigation system. However, the main disadvantage of the navigation system based on GNSS is that it is susceptible to changes in environmental factors, such as the physical structure of the environment, the obstruction of obstacles, adverse weather and signal interruption. INS can obtain the attitude, velocity and position information of an object by integrating angular and linear motion information, however, due to temperature and integration drift, it cannot guarantee accuracy in long-term work. LiDAR can obtain the position of a target point in the surrounding environment by detecting the distance between the target point and the sensor. However, the measurement of distance can be severely affected by the dust generated during the harvesting of crops. In addition, due to the lack of information such as the color and texture of the object, the LiDAR sensor cannot distinguish the type of obstacles. Compared with cameras, the high cost of LiDAR sensors limits their application. Compared with other navigation technologies, machine vision navigation system is increasingly applied to the autonomous navigation of agricultural robots due to its low cost, easy maintenance, wide applicability and high degree of intelligence.

[0003] In recent years, many new methods, new technologies and new platforms related to machine vision have been applied to agricultural robots. Therefore, the vision-based navigation method has become a hot research direction in the current agricultural robot navigation research due to its advantages of rich information, low cost and flexible deployment. With the development of smart agriculture, automatic navigation robots play an increasingly important role in feed feeding, cleaning and inspection of beef cattle farms. Reliable autonomous navigation relies on the accurate perception of robots in complex unstructured environments, and the core task is to perform pixel-level semantic segmentation on navigation paths, livestock and obstacles.

[0004] The beef cattle environment has dynamicity (such as the movement of beef cattle), complexity (various facilities and obstacles exist) and uncertainty (changes in lighting conditions, etc.), and traditional navigation methods are difficult to adapt. Therefore, a semantic segmentation method that can balance multi-scale feature perception, detail preservation and high computational efficiency is needed to ensure the safety and reliability of the navigation robot. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, in order to solve the above problems, the purpose of the present application is to provide a cattle field scene semantic segmentation method, device, equipment and medium based on an improved SegFormer model, which can significantly improve the accuracy and reliability of semantic segmentation in a complex beef cattle field environment, while ensuring the lightweight and real-time performance of the model.

[0006] In order to achieve the above application purpose, the technical scheme adopted by the present application is: In a first aspect, the present application provides a cattle field scene semantic segmentation method, characterized in that it comprises: Obtaining pictures of the central road, the area close to the fence and the obstacle in the cattle field; Inputting the pictures into a cattle field image semantic segmentation model established based on an improved SegFormer network architecture for semantic segmentation processing to complete the identification of the cattle field, wherein the cattle field image semantic segmentation model comprises an encoder and a decoder; The encoder adopts a hierarchical Transformer structure, and a multi-scale hollow attention module is cascaded after each Transformer Block to form a double feature enhancement unit; The decoder comprises a lightweight multi-layer perceptron layer, a hierarchical efficient feature fusion module and a classification head, the lightweight multi-layer perceptron layer is used to unify the channel dimension of the feature maps of different resolutions output by the multi-scale hollow attention module, the hierarchical efficient feature fusion module is used to refine the features of the unified channel dimension feature maps, and the classification head is used to predict the category of each pixel in the feature-refined feature maps.

[0007] In some possible implementations, the training process of the cattle field image semantic segmentation model based on the improved SegFormer network architecture comprises: Constructing a cattle field scene semantic segmentation dataset; Training the cattle field image semantic segmentation model based on the cattle field semantic segmentation dataset, and setting a class weighted cross-entropy loss as a loss function in the training process, when the loss function converges, the trained cattle field image semantic segmentation model is obtained, which is used to complete the identification of the cattle field.

[0008] In some possible implementations, the construction of the cattle field scene semantic segmentation dataset comprises: Using a camera mounted on a robot platform to collect images and videos of the central road, the area close to the fence and the obstacle in the cattle field; Frame processing the videos, and screening out pictures with good lighting and representative scenes; Using a picture labeling tool to finely label the screened images at the pixel level, and the defined categories include railings, feed, roads and cattle; The pictures are cropped to conform to the navigation perspective, and the collected data is allocated according to the training set, the validation set and the test set to construct a semantic segmentation data set of the cattle field.

[0009] In some possible embodiments, the multi-scale hollow attention module is used for enhancement processing of multi-level local features output by the Transformer Block, the multi-scale hollow attention module extracts and fuses multi-scale features through a plurality of hollow convolution branches with different dilated rates, and a hierarchical dilated rate strategy is adopted: The first-stage multi-scale differential attention mechanism enhances the high-resolution feature map, i.e., the original Figure 1 / 4 size, uses a small dilated rate combination [1, 2, 3, 4], and focuses on capturing detailed features such as the edges of the cattle and the textures of the fence; The second-stage multi-scale differential attention mechanism enhances the resolution feature map, i.e., the original Figure 1 / 8 size, uses a medium dilated rate combination [1, 3, 6, 9], retains local details while fusing more extensive context information; The third-stage multi-scale differential attention mechanism enhances the low-resolution feature map, i.e., the original Figure 1 / 16 size, uses a large dilated rate combination [1, 6, 12, 18], realizes modeling of nearly global semantic information, and effectively perceives large objects and scene layout.

[0010] In some possible implementation manners, the Transformer Block is used for multi-level local feature extraction processing on the input feature map and is transmitted to the multi-scale hollow attention module. The Transformer module established based on the multi-scale hollow attention module includes a first Transformer unit, a second Transformer unit, a third Transformer unit, and a fourth Transformer unit. The first Transformer unit is used for attention calculation on an embedding feature vector obtained based on an input picture to obtain a first feature map with a quarter resolution. After the first feature map is processed by a first-stage multi-scale differential attention mechanism enhancement, the first feature map is transmitted to the second Transformer unit and a decoder. The second Transformer unit is used for attention calculation processing on the feature map with the quarter resolution to obtain a feature map with an eighth resolution. After the feature map with the eighth resolution is processed by a second-stage multi-scale differential attention mechanism enhancement, the feature map is transmitted to the third Transformer unit and the decoder. The third Transformer unit is used for attention calculation processing on the feature map with the eighth resolution to obtain a feature map with a sixteenth resolution. After the feature map with the sixteenth resolution is processed by a third-stage multi-scale differential attention mechanism enhancement, the feature map is transmitted to the fourth Transformer unit and the decoder. The fourth Transformer unit is used for attention calculation processing on the feature map with the sixteenth resolution to obtain a feature map with a thirty-second resolution, which is directly transmitted to the decoder.

[0011] In some possible implementation manners, the lightweight multi-layer perception layer is used for linear transformation and nonlinear activation processing on a plurality of feature maps with different resolutions. The lightweight multi-layer perception layer inputs are multi-scale feature maps with four different resolutions, and the feature maps with unified channel dimensions are output by the lightweight multi-layer perception layer.

[0012] In some possible implementation manners, the hierarchical efficient feature fusion module implements channel compression on the feature map with unified channel dimensions output by the lightweight multi-layer perception layer through a 1×1 convolution structure. Then, the global-local fusion module composed of a depth dilated convolution and a Squeeze-and-Excitation structure is used to perform local context modeling and global channel re-labeling on the fusion feature, so as to enhance the expression ability of the fusion feature. The feature map processed by the hierarchical efficient feature fusion module is input to the classification head to perform class prediction on each pixel in the feature map, and finally, a semantic segmentation result consistent with the resolution of the input image is output.

[0013] In a second aspect, the present application further provides a cattle field scene semantic segmentation device, which comprises: The picture acquisition unit is configured to acquire pictures of a central area of a road in a cattle farm, an area close to a fence and an obstacle area; The cattle farm recognition unit is configured to input the pictures into a cattle farm image semantic segmentation model based on an improved SegFormer network architecture for semantic segmentation processing to complete recognition of the cattle farm, wherein the cattle farm image semantic segmentation model comprises an encoder and a decoder. The encoder adopts a hierarchical Transformer structure, and a multi-scale hollow attention module is cascaded after each Transformer Block to form a double-feature enhancement unit. The decoder comprises a lightweight multi-layer perception layer, a hierarchical efficient feature fusion module and a classification head, the lightweight multi-layer perception layer is used for unifying channel dimensions of a plurality of feature maps of different resolutions output by the multi-scale hollow attention module, the hierarchical efficient feature fusion module is used for feature refining of the feature maps after channel dimension unification, and the classification head is used for class prediction of each pixel in the feature maps after feature refining.

[0014] In a third aspect, the present application also provides an electronic device, comprising at least one processor and a memory connected with the processor in communication, wherein the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to execute the method.

[0015] In a fourth aspect, the present application also provides a computer-readable storage medium storing one or more programs, wherein the one or more programs comprise computer instructions for causing a computer to execute the method.

[0016] The present application has the following characteristics due to the above technical solutions. 1. The present application introduces a multi-scale hollow attention (MSDA) module in the first three stages of the encoder, which enables the model to capture rich local details and extensive context information simultaneously through parallel hollow convolution with different inflation rates. The inflation rate is configured differently for different network depths, so that the shallow network focuses on detailed features (which is beneficial for small targets), and the deep network focuses on global semantics, thereby effectively solving the challenge of varying target sizes in the beef cattle farm environment and significantly improving the recognition and segmentation accuracy of key targets such as "cow head" and "feed".

[0017] 2. The present application uses a deformable attention (Deformable Attention) mechanism to replace the standard self-attention mechanism, so that the model can adaptively focus computing resources on key areas with more semantic information in the image, rather than uniformly processing all pixels, making the contour segmentation more accurate.

[0018] 3. The application introduces a dynamic weighted cross-entropy loss function based on the frequency of category pixels, avoids the problem of data imbalance, and enhances the practicality and reliability of the segmentation result.

[0019] 4. The application designs a hierarchical efficient feature fusion module (HEFS) at the decoding end, which is used to process the multi-scale fusion features generated by the MLP decoder. Through channel compression and GLFusion integrating deep dilated convolution and Squeeze-and-Excitation, the segmentation effect of different scale targets (such as large cattle and long railings) is enhanced, and the structure remains lightweight.

[0020] In summary, the application aims to solve the problem of insufficient multi-scale feature modeling and low attention to small targets in the existing SegFormer, and can be widely applied in cattle field scene recognition. BRIEF DESCRIPTION OF DRAWINGS

[0021] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included solely for their for the purpose of illustrating the preferred embodiments and are not to be construed as a limitation of the present application. Throughout the drawings, same reference numerals are used for same components. In the drawings: Figure 1 Flow chart of the improved SegFormer cattle field scene semantic segmentation method of the embodiment of the application; Figure 2 Structure diagram of the cattle field scene semantic segmentation model of the embodiment of the application; Figure 3 Structure diagram of the MSDA mechanism of the embodiment of the application; Figure 4 Module structure diagram of the HEFS of the embodiment of the application; Figure 5 Structure diagram of the electronic device of the embodiment of the application. DETAILED DESCRIPTION

[0022] It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order

[0023] Although the terms first, second, third, etc. can be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms can be only used to distinguish one element, component, region, layer or section from another region, layer or section. Terms such as "first", "second", and other numerical terms when used herein do not imply a sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the example embodiments.

[0024] Spatially relative terms, such as "inner", "outer", "beneath", "below", "lower", "above", "upper", and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. The spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures.

[0025] Aiming at the need for a semantic segmentation method that can balance multi-scale feature perception, detail preservation and high computational efficiency to ensure the safety and reliability of the navigation robot, the application provides a beef field scene semantic segmentation method, device, equipment and medium, which comprises the following steps: acquiring pictures of the road center, the area close to the fence and the obstacle area in the beef field; inputting the pictures into a beef field image semantic segmentation model based on an improved SegFormer network architecture for semantic segmentation processing to complete the identification of the beef field, wherein the beef field image semantic segmentation model comprises an encoder and a decoder; the encoder adopts a hierarchical Transformer structure, and a multi-scale hollow attention module is cascaded after each Transformer Block to form a double feature enhancement unit; the decoder comprises a lightweight multi-layer perceptron layer, a hierarchical efficient feature fusion module and a classification head, the lightweight multi-layer perceptron layer is used for unifying the channel dimension of the feature maps of different resolutions output by the multi-scale hollow attention module, the hierarchical efficient feature fusion module is used for feature refining of the feature maps of the unified channel dimension, and the classification head is used for class prediction of each pixel in the feature maps after feature refining.

[0026] Therefore, by introducing the multi-scale hollow attention module and the deformable attention mechanism, the feature capturing capability of the model for multi-scale targets (especially small targets) is enhanced; at the same time, the class imbalance problem is effectively alleviated by using the class weighted loss function, thereby significantly improving the accuracy and reliability of semantic segmentation in a complex beef field environment.

[0027] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood, and the scope of the present application can be accurately conveyed to those skilled in the art.

[0028] Embodiment one: as shown in the figure, the beef field scene semantic segmentation method based on the improved SegFormer model provided in the embodiment comprises the following steps: Figure 1 S1: constructing a beef field scene semantic segmentation dataset. S1: constructing a beef field scene semantic segmentation dataset.

[0029] In the embodiment, the construction of the beef field scene semantic segmentation dataset comprises the following steps: S11: using a camera mounted on a robot platform to collect images and videos of the road center, the area close to the fence and obstacles in the beef field. The video resolution is 1920x1080, and the frame rate is 30fps.

[0030] S12: Frame processing is performed on the video, and images with good lighting and representative scenes are selected, wherein the images with representative scenes mainly refer to images of cowsheds, passageways, and main channels of feed areas in the preset driving route of the robot.

[0031] S13: Fine pixel-level labeling is performed on the selected images using an image labeling tool such as X-Anylabeling open source tool, and the defined categories include railings, feed, roads, and cows, wherein the pixel-level labeling refers to labeling through X-Anylabeling, mainly polygon labeling, and each pixel in the target area is accurately labeled by drawing irregular shapes, which is suitable for image segmentation and other tasks requiring pixel-level accuracy.

[0032] S14: To ensure compliance with the robot route range, the pictures are cropped to 600x1080 to meet the navigation viewing angle. The collected data is allocated according to the training set: verification set: test set = 70%: 15%: 15%, and a cow farm semantic segmentation data set is constructed.

[0033] S2: As shown in Figure 2 , a cow farm image semantic segmentation model is established based on the improved SegFormer network architecture, the cow farm image semantic segmentation model is trained based on the cow farm semantic segmentation data set, and a class weighted cross entropy loss is set as the loss function in the training process, and when the loss function converges, the trained cow farm image semantic segmentation model is obtained.

[0034] In this embodiment, the cow farm image semantic segmentation model is established based on the improved SegFormer network architecture, including an encoder and a decoder.

[0035] In this embodiment, the encoder adopts a hierarchical Transformer structure, and a multi-scale hollow attention (MSDA) module is cascaded after each Transformer Block to form a double feature enhancement unit.

[0036] The Transformer module established by the Multi-Scale Dilated Attention (MSDA) module includes a first Transformer unit, a second Transformer unit, a third Transformer unit, and a fourth Transformer unit. The first Transformer unit performs attention calculations on the embedded feature vector (x) to obtain a first feature map at a quarter resolution. This first feature map is enhanced by a first-stage multi-scale differential attention mechanism and then transmitted to the second Transformer unit and the decoder. The second Transformer unit performs attention calculations on the quarter-resolution feature map to obtain a feature map at a one-eighth resolution. This feature map is enhanced by a second-stage multi-scale differential attention mechanism and then transmitted to the third Transformer unit and the decoder. The third Transformer unit performs attention calculations on the one-eighth resolution feature map to obtain a feature map at a one-sixteenth resolution. This feature map is enhanced by a third-stage multi-scale differential attention mechanism and then transmitted to the fourth Transformer unit and the decoder. The fourth Transformer unit performs attention calculations on the one-sixteenth resolution feature map to obtain a feature map at a one-thirty-second resolution and directly transmits it to the decoder.

[0037] Specifically, the embedded feature vector refers to the sequence data obtained after the original image information has been transformed through patching and projection, and can be processed by the Transformer unit. Taking the mit_b0 model and a 512*512 image as an example: Input image: size 512*512, number of channels 3 (RGB); Patch embedding (OverlapPatchEmbed1): uses a 7*7 convolutional kernel with a stride of 4, projecting the 3 input channels onto embed_dim=32 output channels. The output feature map size is 128*128, and the number of channels is 32. Serialization (Flatten & Transpose): The feature map is flattened (x.flatten(2)) from B*32*128*128 to B*32*16384, and then transposed (transpose(1,2)) to form a tensor of B*16384*32, which is the embedded feature vector. It can be regarded as a sequence of 16384 feature vectors, each with a dimension of 32. The above 16384 32-dimensional vectors are the input of the first Transformer unit. The original input of the encoder is an image. The embedded feature vector is a high-dimensional representation of the image obtained after block division and linear projection, which can be processed by the Transformer. It is another mathematical expression of the image and contains local information of the image.

[0038] Specifically, the Multi-Scale Dilated Attention (MSDA) module extracts and fuses multi-scale features through multiple dilated convolutional branches with different dilation rates, employing a hierarchical dilation rate strategy: Stage 1: Multi-scale differential attention mechanism enhances the processing of high-resolution feature maps (original) Figure 1 ( / 4 size), using a small dilation ratio combination [1,2,3,4], focusing on capturing detailed features such as cattle edges and fence textures.

[0039] The second stage (Stage 2) utilizes a multi-scale differential attention mechanism to enhance the processing resolution of feature maps (original). Figure 1 ( / 8 size), using the dilation ratio combination [1,3,6,9], to integrate broader contextual information while preserving local details.

[0040] The third stage (Stage 3) enhances the processing of low-resolution feature maps through a multi-scale differential attention mechanism. Figure 1 ( / 16 size), using a large dilation ratio combination [1,6,12,18], to achieve near-global semantic information modeling, effectively perceiving large objects and scene layout.

[0041] Furthermore, each of the first to third Transformer units includes a Transformer Block and an MSDA module. The Transformer Block performs multi-level local feature extraction on the input feature map and transmits it to the MSDA module. The MSDA module enhances the multi-level local features output by the Transformer Block. The structure of the MSDA module is as follows: Figure 3 As shown, K (Key) is the key vector, serving as the "identifier" or "index" of the query, and V (Value) is the value vector, storing the actual "content information." The core component is Sliding Window Dilated Attention, which uses a sliding window and dilation rate (r) to control the attention receptive field. The sliding window divides the input feature map into multiple local regions, calculating attention within each region, thus reducing computation while utilizing local prior information. Different dilation rates can acquire contextual information at different scales. After concatenating the extracted multi-scale feature maps, a Linear layer is used for fusion and dimensionality reduction. Finally, a residual connection is made with the module's input feature map (the input feature map refers to the intermediate feature map output from each stage of the Mix Vision Transformer (MiT) backbone network, which has already been processed by the Transformer Block), outputting an enhanced feature map. The enhanced feature map is a new feature map output by the MSDA module, incorporating multi-scale contextual information. For example, assuming the input is Stage... Feature map C1 (32*128*128 pixels) contains local edge details of the cattle. The enhanced feature C'1 is also 32*128*128 pixels. Since the weight map is generated from both global (expansion rate d=4) and local (expansion rate d=1) information, higher values ​​(close to 1) are assigned to regions more important for segmentation (e.g., the cattle's head). Lower values ​​are assigned to background regions less important for segmentation (e.g., feed). Ultimately, the enhanced feature C'1 strengthens discriminative local and cross-regional correlation information, resulting in stronger feature representation and effectively addressing variations in the scale and shape of targets such as cattle and fences. This design allows the model to process information from different receptive fields in parallel, effectively handling changes in target scale.

[0042] Furthermore, the self-attention mechanism in the Transformer Block of each stage is replaced with Deformable Attention to adaptively focus on key regions in the image, enhancing the model's multi-scale feature modeling ability and robustness to target deformation, thus satisfying the image segmentation requirements for cattle farm environments.

[0043] Specifically, such as Figure 3 As shown, the original efficient self-attention (Self-Attn) is replaced with multi-scale deformable attention. This mechanism dynamically predicts sampling offsets and attention weights, enabling each query point to adaptively focus on key regions on the multi-scale feature map (key regions are defined as: "feature sampling points that have the most discriminative feature aggregation for the current query point and contribute the least to the loss of the final semantic segmentation, i.e., the best performance"). The improved model significantly enhances the attention to small target regions, forming a more continuous attention response on target edges and internal structures. This design has a significant advantage in computational efficiency, especially suitable for high-resolution image processing, while maintaining robustness to object deformation and scale changes.

[0044] In this embodiment, the decoder includes a lightweight multilayer perceptron layer (MLP) and a hierarchical high-efficiency feature fusion module (HEFS).

[0045] Furthermore, a lightweight multilayer perceptron (MLP) layer is used to perform linear transformations of the channel dimensions on feature maps from different resolutions of the encoder, and to achieve spatial scale unification through upsampling operations. Subsequently, the unified feature maps are stitched together along the channel dimensions to obtain a fused feature map containing multi-scale information.

[0046] The lightweight multilayer perceptron (MLP) layer is used to perform linear transformations and non-linear activations on several feature maps of different resolutions. The MLP layer receives four multi-scale feature maps of different resolutions. These feature maps are extracted by the backbone network (MiT) and enhanced by the MSDA module (except for Stage 4 features) to unify the dimensionality of the feature maps at different resolutions. The MLP outputs a new set of feature maps with unified channel dimensions, preparing for the next steps of upsampling and feature concatenation.

[0047] Furthermore, such as Figure 4 As shown, the Hierarchical Efficient Feature Fusion (HEFS) module is used to refine the fused feature map. This module first achieves channel compression using a 1×1 convolutional structure; then, it employs a Global-Local Fusion (GLFusion) module, consisting of depthwise dilated convolutions and a squeeze-and-excitation structure, to perform local context modeling and global channel recalibration on the fused features, thereby enhancing the expressive power of the fused features. The feature map processed by the HEFS module is input to the Segmentation Head, which includes a 1×1 convolutional structure for class prediction of each pixel in the feature map, ultimately outputting a semantic segmentation result with the same resolution as the input image.

[0048] In this embodiment, the loss function used during training is the class-weighted cross-entropy loss function:

[0049] Among them, category-weighted cross-entropy loss In standard cross-entropy loss Based on this, weight factors were added to each category. .

[0050] Furthermore, the standard cross-entropy loss function :

[0051]

[0052] in, This represents the total number of pixels after downsampling. H W represents the height of the feature map, which is the number of pixels in the vertical direction, and W represents the width of the feature map, which is the number of pixels in the horizontal direction. C For the number of categories, ∈{0,1} represents the first... i Pixels in category c The actual label (one-hot encoded) on it. This represents the corresponding probability predicted by the model. The loss is backpropagated through the AdamW optimizer (initial learning rate 6e-5) with a "poly" decay strategy (factor 1.0), driving the hierarchical Transformer encoder (MiT) and MLP decoder to learn discriminative features.

[0053] Specifically, to address the data imbalance problem of sparse cattle targets in cattle farm image segmentation, a class-weighted cross-entropy loss is introduced on top of the standard loss. This improvement works by dynamically assigning class weights. This strengthens the model's focus on the minority class (cattle) and optimizes the class imbalance problem.

[0054] S3: Based on the trained cattle farm image semantic segmentation model, perform cattle farm image semantic segmentation processing on the image to be identified, and complete the identification of the cattle farm. The cattle farm identification results include fences, feed, roads and cattle.

[0055] Example 2: Following the above-described Example 1, which provides a multi-scale semantic segmentation method for a cattle farm scene using an improved SegFormer model, this example provides a device for multi-scale semantic segmentation of a cattle farm scene using an improved SegFormer model. The device provided in this example can implement the multi-scale semantic segmentation method for a cattle farm scene using an improved SegFormer model as described in Example 1. This device can be implemented through software, hardware, or a combination of both. For ease of description, this example is described by dividing the functionality into various units. Of course, in implementation, the functions of each unit can be implemented in one or more software and / or hardware components. For example, the device may include integrated or separate functional modules or units to execute the corresponding steps in the methods of Example 1. Since the device in this example is basically similar to the method example, the description process of this example is relatively simple. For relevant details, please refer to the description in Example 1. The example of the multi-scale semantic segmentation device for a cattle farm scene using an improved SegFormer model provided by this invention is merely illustrative.

[0056] Specifically, this embodiment also provides a semantic segmentation device for a cattle farm scene, the device comprising: The image acquisition unit is configured to acquire images of the area in the center of the road within the cattle farm, near the fence and obstacles; The cattle farm identification unit is configured to input images into a cattle farm image semantic segmentation model based on an improved SegFormer network architecture for semantic segmentation processing, thereby completing the identification of the cattle farm. The cattle farm image semantic segmentation model includes an encoder and a decoder. The encoder adopts a hierarchical Transformer structure, with a multi-scale dilated attention module cascaded after each Transformer Block to form a dual feature enhancement unit; The decoder includes a lightweight multilayer perceptron layer, a hierarchical efficient feature fusion module, and a classification head. The lightweight multilayer perceptron layer is used to unify the channel dimensions of several feature maps of different resolutions output by the multi-scale dilated attention module. The hierarchical efficient feature fusion module is used to refine the features of the feature maps with unified channel dimensions. The classification head predicts the category of each pixel in the feature map after feature refinement.

[0057] Example 3: This example provides an electronic device corresponding to the multi-scale semantic segmentation method for cattle farm scenes based on the improved SegFormer model provided in Example 1. The electronic device can be an electronic device for client use, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Example 1.

[0058] like Figure 5As shown, the electronic device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the method of Embodiment 1. The implementation principle and technical effects are similar to those of Embodiment 1, and will not be repeated here. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computing device on which the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0059] In a preferred embodiment, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), and optical discs.

[0060] In a preferred embodiment, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and is not limited thereto.

[0061] Example 4: This example provides a computer-readable storage medium for storing one or more programs, the one or more programs including computer instructions, which, when executed by a computer, cause the computer to perform the method provided in Example 1 above.

[0062] In a preferred embodiment, the computer-readable storage medium may be a tangible device for holding and storing instructions executable, such as, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. The computer-readable storage medium stores computer program instructions that cause a computer to perform the method provided in Embodiment 1 above.

[0063] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0065] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In the description of this specification, the terms "a preferred embodiment," "furthermore," "specifically," "in this embodiment," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semantic segmentation method for a cattle farm scene, characterized in that, include: Obtain images of the area in the center of the road within the cattle farm, near the fences and obstacles; The image is input into a semantic segmentation model for cattle farm images based on an improved SegFormer network architecture for semantic segmentation processing to complete the identification of cattle farms. The semantic segmentation model for cattle farm images includes an encoder and a decoder. The encoder adopts a hierarchical Transformer structure, with a multi-scale dilated attention module cascaded after each Transformer Block to form a dual feature enhancement unit; The decoder includes a lightweight multilayer perceptron layer, a hierarchical efficient feature fusion module, and a classification head. The lightweight multilayer perceptron layer is used to unify the channel dimensions of several feature maps of different resolutions output by the multi-scale dilated attention module. The hierarchical efficient feature fusion module is used to refine the features of the feature maps with unified channel dimensions. The classification head predicts the category of each pixel in the feature map after feature refinement.

2. The semantic segmentation method for a cattle farm scene according to claim 1, characterized in that, The training process for establishing a semantic segmentation model for cattle farm images based on the improved SegFormer network architecture includes: Construct a semantic segmentation dataset for cattle farm scenes; A semantic segmentation model for cattle farm images is trained based on a cattle farm semantic segmentation dataset. A category-weighted cross-entropy loss is set as the loss function during the training process. When the loss function converges, the trained semantic segmentation model for cattle farm images is obtained, which can be used to identify cattle farms.

3. The semantic segmentation method for a cattle farm scene according to claim 2, characterized in that, Construct a semantic segmentation dataset for cattle farm scenes, including: Cameras equipped with robotic platforms are used to collect images and videos of the central area of ​​the roads within the cattle farm, as well as areas near fences and obstacles. The video is segmented into frames, and images with good lighting and representative scenes are selected. The selected images were annotated with fine pixel-level annotations using an image annotation tool, and the defined categories included fences, feed, roads, and cattle. The images were cropped to fit the navigation viewpoint, and the collected data were distributed into training, validation, and test sets to construct the cattle farm semantic segmentation dataset.

4. The semantic segmentation method for a cattle farm scene according to claim 3, characterized in that, The multi-scale dilated attention module is used to enhance the multi-level local features output by the Transformer Block. It extracts and fuses multi-scale features through multiple dilated convolutional branches with different dilation rates, employing a hierarchical dilation rate strategy. The first stage uses a multi-scale differential attention mechanism to enhance the processing of high-resolution feature maps, which are 1 / 4 the size of the original image. It uses a small dilation rate combination [1,2,3,4] to focus on capturing detailed features such as cattle edges and fence textures. The second stage uses a multi-scale differential attention mechanism to enhance the processing of the resolution feature map, which is 1 / 8 the size of the original image. It uses a dilation rate combination [1,3,6,9] to preserve local details while incorporating broader contextual information. The third stage uses a multi-scale differential attention mechanism to enhance the processing of low-resolution feature maps, which are 1 / 16 the size of the original image. It uses a large dilation rate combination [1,6,12,18] to achieve near-global semantic information modeling and effectively perceive large objects and scene layout.

5. The semantic segmentation method for a cattle farm scene according to claim 4, characterized in that, The Transformer Block performs multi-level local feature extraction on the input feature map and transmits it to the multi-scale dilated attention module. The Transformer module built upon the multi-scale dilated attention module includes a first Transformer unit, a second Transformer unit, a third Transformer unit, and a fourth Transformer unit. The first Transformer unit performs attention calculation on the embedded feature vectors obtained from the input image to obtain a first feature map at one-quarter resolution. This first feature map is then enhanced by a first-stage multi-scale differential attention mechanism and transmitted to the second Transformer unit and the decoder. The second Transformer... The first Transformer unit performs attention calculations on the 1 / 4 resolution feature map to obtain a 1 / 8 resolution feature map. This feature map is then enhanced by a second-stage multi-scale differential attention mechanism and transmitted to the third Transformer unit and the decoder. The second Transformer unit performs attention calculations on the 1 / 8 resolution feature map to obtain a 1 / 6 resolution feature map. This feature map is then enhanced by a third-stage multi-scale differential attention mechanism and transmitted to the fourth Transformer unit and the decoder. The third Transformer unit performs attention calculations on the 1 / 6 resolution feature map to obtain a 1 / 32 resolution feature map and transmits it directly to the decoder.

6. The semantic segmentation method for a cattle farm scene according to claim 3, characterized in that, The lightweight multilayer perceptron layer is used to perform linear transformations and nonlinear activation processing on several feature maps of different resolutions. The input of the lightweight multilayer perceptron layer is four multi-scale feature maps of different resolutions, and the output of the lightweight multilayer perceptron layer is a feature map with unified channel dimensions.

7. The semantic segmentation method for a cattle farm scene according to claim 6, characterized in that, The hierarchical efficient feature fusion module compresses the feature maps with uniform channel dimensions at the output of the lightweight multilayer perceptron layer using a 1×1 convolutional structure. Subsequently, a global-local fusion module consisting of depth dilated convolution and a squeeze-and-excitation structure is used to perform local context modeling and global channel recalibration on the fused features, thereby enhancing the expressive power of the fused features. The feature maps processed by the hierarchical efficient feature fusion module are input to the classification head to predict the category of each pixel in the feature maps, and finally output a semantic segmentation result with the same resolution as the input image.

8. A semantic segmentation device for a cattle farm scene, characterized in that, The device includes: The image acquisition unit is configured to acquire images of the area in the center of the road within the cattle farm, near the fence and obstacles; The cattle farm identification unit is configured to input images into a cattle farm image semantic segmentation model based on an improved SegFormer network architecture for semantic segmentation processing, thereby completing the identification of the cattle farm. The cattle farm image semantic segmentation model includes an encoder and a decoder. The encoder adopts a hierarchical Transformer structure, with a multi-scale dilated attention module cascaded after each Transformer Block to form a dual feature enhancement unit; The decoder includes a lightweight multilayer perceptron layer, a hierarchical efficient feature fusion module, and a classification head. The lightweight multilayer perceptron layer is used to unify the channel dimensions of several feature maps of different resolutions output by the multi-scale dilated attention module. The hierarchical efficient feature fusion module is used to refine the features of the feature maps with unified channel dimensions. The classification head predicts the category of each pixel in the feature map after feature refinement.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include computer instructions for causing a computer to perform the method according to any one of claims 1-7.

Images