Cow behavior recognition and interpretation method based on deep learning image description

Abstract

The invention provides a cattle behavior recognition and interpretation method based on deep learning image description, and the method develops a special image description data set covering diversified scenes and a complete behavior pedigree according to the actual demands of cattle breeding. The data set labeling information completely comprises behavior characteristics, environment information and spatial relations, the pertinence and integrity of data are remarkably improved, a solid data basis is provided for model training, and the defect that an existing general data set is insufficient in applicability in the scene is overcome. Meanwhile, through deep fusion of a computer vision technology and a natural language processing technology and based on a deep learning image description model, direct mapping from a cattle behavior image to natural language description containing environment information is realized for the first time; the problem that the existing method can only identify the behavior and cannot express the relationship between the behavior and the breeding scene is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence in animal husbandry, and in particular to a method for recognizing and interpreting cattle behavior based on deep learning image description. Background Technology

[0002] As my country's livestock farming industry continues to develop towards large-scale and intensive operations, traditional manual farming management methods are no longer sufficient to meet the demands of industry upgrading. Smart ranches are gradually becoming an important direction for industry development. In a smart ranch system, monitoring the behavioral status of cattle is a crucial link in achieving precise feeding, disease early warning, estrus detection, and reproductive management. Currently, the industry has begun to apply deep learning-based computer vision technology for cattle behavior recognition, mainly based on image classification, object detection, and image segmentation methods. This extracts behavioral features from image data to identify basic behaviors such as standing, lying down, and feeding, as well as detecting specific diseases such as hoof slippage and lameness in dairy cows. However, these methods still have several significant limitations: The lack of ability to identify correlations between different farming scenarios is a significant drawback: the same behavior often reflects different meanings for machine management of cattle status in different farming scenarios. Existing methods mostly focus on the identification of the cattle's visual features and have not optimized models for different scenarios such as pens, pastures, watering areas, and feeding areas. As a result, they cannot effectively and accurately describe the relationship between cattle and their surrounding environment, leading to a lack of specific scenario support for behavior recognition results. For example, it is difficult to distinguish between behaviors with different management implications, such as "grazing in the pasture" and "grazing in the pen," or "drinking by the water trough" and "head-down actions in other locations." Furthermore, the behavioral descriptions lack semantic representation: existing technologies typically output single-category labels or numerical features, failing to describe behaviors and their environmental context in intuitive natural language. This makes it difficult for ranch managers, especially non-technical personnel, to quickly understand and utilize the identification results, hindering the timeliness and effectiveness of management decisions. There is also insufficient compatibility with livestock data platforms: existing methods often output fixed and uniform data formats, lacking specific adaptation designs for existing smart ranch data management systems (such as traceability platforms and livestock management platforms). This makes it difficult to directly integrate the identification results into the system for storage, analysis, and application, resulting in data silos. Finally, there is a lack of dedicated datasets: currently available datasets are mostly general image datasets or cattle image sets containing only limited behavioral categories. There is a lack of dedicated image description datasets covering diverse livestock scenarios (such as pens and pastures) and complete behavioral profiles, with environmental association information. This limits the training sufficiency, recognition accuracy, and scene generalization ability of deep learning models. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a method for recognizing and interpreting bovine behavior based on deep learning image description.

[0004] To achieve the above objectives, the technical solution of this invention is as follows: In a first aspect, the present invention provides a method for recognizing and interpreting bovine behavior based on deep learning image description, the method comprising: Multiple original images from cattle farming scenarios are acquired, and each original image is annotated with natural language descriptions to form a cattle behavior image description dataset. The cow behavior image to be identified is input into a trained cow behavior image model to extract the corresponding natural language description text; the trained cow behavior image model is trained based on the cow behavior image description dataset; The natural semantic description text is segmented to extract keywords related to cattle behavior, breed, scene, and location. According to the preset behavior-environment-space cattle-specific mapping rules, the extracted keywords are converted into structured cattle behavior data, which includes behavior codes, breed codes, scene codes, location codes, status codes and timestamps. The structured cattle behavior data is standardized and encapsulated into JSON data format, then sent to the intelligent livestock farming data platform through the interface adaptation module to complete the data connection.

[0005] In some embodiments, the trained cattle behavior image model is trained through the following steps: The CNN-RNN encoder-decoder framework is used as the initial model, where the CNN is used to extract visual features of cattle behavior images and the RNN is used to generate corresponding natural language description text. At the encoder end, an improved CBAM attention module is embedded after the 2nd to 4th stage convolutional layers of the ResNet50 network structure; the improved CBAM attention module uses the attention module to strengthen the feature channel weights of key body parts of the cow, and the spatial attention module is used to focus on key areas of cow behavior. On the decoder side, when introducing an attention mechanism based on RNN, the attention weight allocation strategy is optimized for the temporal correlation of cow behavior; The cattle behavior image description dataset is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The initial model is trained and its parameters are optimized using the training set and the validation set to obtain the trained cattle behavior image model.

[0006] In some embodiments, the cattle farming scene includes at least one of pen, pasture, watering area, and feeding area; the number of original images is not less than 2,000, and the original images cover different lighting conditions, stocking densities, and different breeds of cattle.

[0007] In some embodiments, when the natural language description annotation is performed on each of the original images, the annotation content includes cattle behavioral state features, breeding environment features, and spatial relationship between cattle and facility environment; the cattle behavioral state features include action type and posture details; the breeding environment features include breeding scene and surrounding facilities; the spatial relationship includes the positional relationship between cattle and facility environment.

[0008] In some embodiments, the key areas of bovine behavior include the mouth when feeding, the junction of the head and the water trough when drinking, and the hooves when walking; the temporal correlation of bovine behavior includes the continuous movements of the limbs when walking, and the process of lowering the head, chewing, and raising the head when feeding.

[0009] Secondly, this invention provides a bovine behavior recognition and interpretation system based on deep learning image description, the system comprising: The acquisition module is used to acquire multiple original images in the cattle breeding scenario and to perform natural language description annotation on each of the original images to form a cattle behavior image description dataset. The extraction module is used to input the cow behavior image to be identified into the trained cow behavior image model and extract the corresponding natural semantic description text; the trained cow behavior image model is trained based on the cow behavior image description dataset; The word segmentation module is used to segment the natural semantic description text and extract keywords related to cattle behavior, breed, scene, and location. The conversion module is used to convert the extracted keywords into structured cattle behavior data according to the preset behavior-environment-space cattle-specific mapping rules. The structured cattle behavior data includes behavior codes, breed codes, scene codes, location codes, status codes and timestamps. The standardization module is used to standardize the structured cattle behavior data and encapsulate it into JSON data format. The data is then sent to the intelligent livestock farming data platform through the interface adaptation module to complete the data connection.

[0010] Thirdly, the present invention provides an electronic device, comprising: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the cattle behavior recognition and interpretation method based on deep learning image description as described in any one of the first aspects.

[0011] Fourthly, the present invention provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the cattle behavior recognition and interpretation method based on deep learning image description as described in any one of the first aspects.

[0012] This invention provides a method for cattle behavior recognition and interpretation based on deep learning image description. Through the deep integration of computer vision and natural language processing technologies, it constructs an end-to-end mapping model from image to text. While recognizing the cattle's own behavior, it accurately expresses the spatial and semantic relationships between the cattle and their environment, such as pens and pastures, overcoming the limitations of existing technologies that separate behavior from the environment. For the complex scenarios of cattle farming, a dedicated image description dataset covering multiple scenarios, behaviors, and conditions has been developed. Its annotation information includes behavior, environment, and spatial relationship information, fundamentally solving the problems of weak targeting and incomplete information in general datasets. This provides a high-quality data foundation for model training and overcomes the shortcomings of existing general datasets in this scenario. By formulating dedicated mapping rules and standardized processing procedures tailored to farming operations, the natural language descriptions output by the model can be automatically converted into structured behavioral data that the platform can directly obtain. This directly adapts to the input specifications of the intelligent farming data platform, achieving seamless integration with the intelligent farming system and effectively solving the problems of difficulty in connecting recognition results with pasture management systems and the inability to directly apply data in existing technologies. An attention mechanism was specifically integrated into the encoder-decoder framework, and the model structure was optimized specifically for cattle behavior, making it more focused on the key features and temporal logic of cattle behavior. This significantly improved the accuracy and real-time performance of behavior description, while ensuring the processing efficiency of a single image, meeting the needs of high-accuracy and high-efficiency real-time monitoring and analysis in pasture environments. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a method for recognizing and interpreting cattle behavior based on deep learning image description provided in an embodiment of the present invention; Figure 2 This is a flowchart of another method for recognizing and interpreting cattle behavior based on deep learning image description provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the encoder-decoder model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the behavior interpretation process provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] In the following description, references to "some embodiments" refer to a subset of all possible embodiments; however, it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the invention have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of the invention pertain. The terminology used in the embodiments of the invention is for the purpose of describing the embodiments of the invention only and is not intended to limit the invention.

[0016] The following describes exemplary applications of the deep learning image description-based cow behavior recognition and interpretation device according to embodiments of the present invention. This device can be implemented as a terminal or a server. In one implementation, it can be implemented as a terminal such as a laptop, tablet, desktop computer, or mobile device. In another implementation, it can also be implemented as a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and server can be directly or indirectly connected via wired or wireless communication, without limitation in the embodiments of the present invention. The following will describe exemplary applications of the deep learning image description-based cow behavior recognition and interpretation device as a server.

[0017] This invention provides a method for recognizing and interpreting cow behavior based on deep learning image descriptions. See [link to relevant documentation]. Figure 1 , Figure 1 This is a flowchart illustrating a method for recognizing and interpreting cow behavior based on deep learning image description provided in an embodiment of the present invention. Figure 1 The steps shown are explained.

[0018] Step S110: Obtain multiple original images in the cattle farming scenario, and perform natural language description annotation on each of the original images to form a cattle behavior image description dataset.

[0019] In some implementations, a cattle farming scenario refers to the overall composition of various specific spaces and environments used for the feeding, breeding, and growth of cattle. It possesses the basic conditions necessary for the survival and activity of cattle and is under artificial or semi-artificial control. In actual farming, the scenario can be further divided according to dimensions such as farming model, functional zoning, and environmental type, serving as the basic environmental carrier for cattle behavior.

[0020] In some implementations, raw images refer to unprocessed, original visual data carriers directly acquired through image acquisition devices in cattle farming scenarios. These images contain core information such as the cattle's behavioral state, breed characteristics, surrounding environment, and spatial location. For example, these could be images of dairy cows grazing, resting, or walking captured by high-definition cameras installed in cattle sheds; images of yak herds migrating captured by drones over plateau fields; or images of calves nursing captured by handheld cameras.

[0021] In some implementations, natural language semantic description annotation refers to the precise, complete, and standardized descriptive annotation operation on the core information in the original image. The annotation result needs to accurately map the visual information in the image and establish a connection between the original image and the semantic information. This is the core process for constructing a dataset of cattle behavior image descriptions. For example, for an original image of Simmental cattle standing and resting in an open-air enclosure, the annotation could be "Simmental cattle, in the center of the open-air enclosure, standing and resting on all fours."

[0022] In some implementations, the cattle behavior image description dataset refers to a structured data set consisting of multiple sets of original images and natural language description annotations, which serves as the core training sample carrier for training cattle behavior image models.

[0023] Step S120: Input the cow behavior image to be identified into the trained cow behavior image model and extract the corresponding natural semantic description text; the trained cow behavior image model is trained based on the cow behavior image description dataset.

[0024] In some implementations, the cattle behavior images to be identified refer to cattle behavior images that are collected in real time or imported in batches in cattle farming scenarios, have not yet been labeled with any natural language descriptions, and whose core information (behavior, breed, scene, location, etc.) needs to be identified through a trained cattle behavior image model.

[0025] In some implementations, the natural language description text refers to the natural language text output by the trained cattle behavior image model after analyzing and processing the input cattle behavior image to be identified. This text is used to characterize the core information of the image (cattle behavior, breed, scene, location, etc.). Its format is consistent with the natural language description annotation text in the cattle behavior image description dataset to ensure compatibility with subsequent steps such as word segmentation and keyword extraction.

[0026] Step S130: Perform word segmentation on the natural semantic description text to extract cattle behavior keywords, breed keywords, scene keywords and location keywords.

[0027] In some implementations, word segmentation refers to the use of word segmentation algorithms from the field of natural language processing, such as Jieba segmentation and HanLP segmentation, combined with a professional terminology dictionary in the cattle breeding field, to split the target natural semantic description text into independent words or phrases according to semantic logic. This process can extract basic units with independent semantics, providing a prerequisite for subsequent classification and extraction of cattle behavior keywords, breed keywords, etc.

[0028] For example, the natural semantic description text "Holstein cows are eating in front of the trough in the enclosed cowshed" is segmented into the following words: Holstein cows, enclosed cowshed, trough, eating.

[0029] In some implementations, cattle behavior keywords refer to words or phrases extracted from word segmentation results that characterize specific cattle behaviors or activity states, and are key information describing the cattle's physiological state and activity needs. Examples include feeding, lying down, standing, walking, drinking, ruminating, scratching, kicking, running, resting, and waiting to be milked.

[0030] In some implementations, breed keywords refer to words or phrases extracted from word segmentation results that characterize specific cattle breed types. They are key information for distinguishing different cattle breeds and are directly related to the breed attributes of cattle, such as dairy cattle and beef cattle.

[0031] In some implementations, scene keywords refer to words or phrases extracted from word segmentation results that characterize the type of cattle farming scene in which the cattle are located. They are key information for distinguishing different farming environments and correspond to the core environmental features in the cattle farming scene. Examples include enclosed cattle sheds, open-air fences, mountain pasture areas, milking parlors, and watering areas.

[0032] In some implementations, location keywords refer to words or phrases extracted from word segmentation results that characterize the specific spatial location of a cow within the current farm. These are key information for locating the relative position of a cow within the scene and are usually associated with specific facilities or areas within the scene. Examples include: left side of the feed trough, in front of the feed trough, center of the fence, entrance to the milking parlor, next to the watering area, and inside the lying area.

[0033] Step S140: Based on the preset behavior-environment-space cattle-specific mapping rules, the extracted keywords are converted into structured cattle behavior data, which includes behavior codes, breed codes, scene codes, location codes, status codes, and timestamps.

[0034] In some implementations, the preset behavior-environment-space cattle-specific mapping rules refer to a set of rules pre-customized for cattle farming scenarios, used to convert unstructured keyword information into structured coded data. These rules are specific to the cattle farming scenario, requiring the establishment of a unique correspondence between keywords and codes, and also covering the determination rules for status codes and the time schedule rules for timestamps, to ensure that the converted structured cattle behavior data meets the standardized management requirements of the cattle farming industry.

[0035] In some implementations, structured cattle behavior data refers to standardized data with a fixed data structure and uniform format, formed by converting various extracted keywords based on cattle-specific mapping rules of behavior-environment-space.

[0036] Step S150: The structured cattle behavior data is standardized and encapsulated into JSON data format, and sent to the intelligent breeding data platform through the interface adaptation module to complete the data connection.

[0037] This invention provides a method for cattle behavior recognition and interpretation based on deep learning image description. Through the deep integration of computer vision and natural language processing technologies, it constructs an end-to-end mapping model from image to text. While recognizing the cattle's own behavior, it accurately expresses the spatial and semantic relationships between the cattle and their environment, such as pens and pastures, overcoming the limitations of existing technologies that separate behavior from the environment. For the complex scenarios of cattle farming, a dedicated image description dataset covering multiple scenarios, behaviors, and conditions has been developed. Its annotation information includes behavior, environment, and spatial relationship information, fundamentally solving the problems of weak targeting and incomplete information in general datasets. This provides a high-quality data foundation for model training and overcomes the shortcomings of existing general datasets in this scenario. By formulating dedicated mapping rules and standardized processing procedures tailored to farming operations, the natural language descriptions output by the model can be automatically converted into structured behavioral data that the platform can directly obtain. This directly adapts to the input specifications of the intelligent farming data platform, achieving seamless integration with the intelligent farming system and effectively solving the problems of difficulty in connecting recognition results with pasture management systems and the inability to directly apply data in existing technologies. An attention mechanism was specifically integrated into the encoder-decoder framework, and the model structure was optimized specifically for cattle behavior, making it more focused on the key features and temporal logic of cattle behavior. This significantly improved the accuracy and real-time performance of behavior description, while ensuring the processing efficiency of a single image, meeting the needs of high-accuracy and high-efficiency real-time monitoring and analysis in pasture environments.

[0038] In some implementations, the trained image model of cattle behavior is trained through the following steps: The CNN-RNN encoder-decoder framework is used as the initial model, where the CNN is used to extract visual features of cattle behavior images and the RNN is used to generate corresponding natural language description text. At the encoder end, an improved CBAM attention module is embedded after the 2nd to 4th stage convolutional layers of the ResNet50 network structure; the improved CBAM attention module uses the attention module to strengthen the feature channel weights of key body parts of the cow, and the spatial attention module is used to focus on key areas of cow behavior. On the decoder side, when introducing an attention mechanism based on RNN, the attention weight allocation strategy is optimized for the temporal correlation of cow behavior; The cattle behavior image description dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:1. The initial model is trained and its parameters are optimized using the training set and the validation set to obtain the trained cattle behavior image model.

[0039] In this invention, an attention mechanism is specifically integrated into the encoder-decoder framework, and the model structure is specifically optimized to focus more on the key features and temporal logic of cattle behavior. This optimization significantly improves the accuracy of behavior description while ensuring the processing efficiency of a single image, meeting the real-time monitoring and analysis needs in a pasture environment.

[0040] In some embodiments, the cattle farming scene includes at least one of pen, pasture, watering area, and feeding area; the number of original images is not less than 2,000, and the original images cover different lighting conditions, stocking densities, and different breeds of cattle.

[0041] In some embodiments, when the natural language description annotation is performed on each of the original images, the annotation content includes cattle behavioral state features, breeding environment features, and spatial relationship between cattle and facility environment; the cattle behavioral state features include action type and posture details; the breeding environment features include breeding scene and surrounding facilities; the spatial relationship includes the positional relationship between cattle and facility environment.

[0042] In this invention, a dedicated image description dataset covering diverse scenarios and a complete behavioral spectrum was developed to meet the actual needs of cattle farming. The dataset's annotation information comprehensively includes behavioral features, environmental information, and spatial relationships, significantly improving the data's relevance and completeness. This provides a solid data foundation for model training and overcomes the shortcomings of existing general datasets in terms of applicability in this scenario.

[0043] In some embodiments, the key areas of bovine behavior include the mouth when feeding, the junction of the head and the water trough when drinking, and the hooves when walking; the temporal correlation of bovine behavior includes the continuous movements of the limbs when walking, and the process of lowering the head, chewing, and raising the head when feeding.

[0044] In this invention, by formulating dedicated mapping rules that fit the needs of aquaculture management and combining them with data standardization processing, the natural language descriptions output by the model can be automatically converted into structured behavioral data, thereby directly adapting to the input specifications of the intelligent aquaculture data platform. This effectively solves the problems of difficulty in connecting the recognition results with the ranch management system and difficulty in directly applying the data in the prior art.

[0045] The following will describe an exemplary application of the embodiments of the present invention in a practical application scenario.

[0046] This invention proposes a method for recognizing and interpreting cattle behavior based on deep learning image description. This method employs a comprehensive technical solution encompassing image acquisition and dataset construction, deep learning image description model training, semantic interpretation of cattle behavior, and data integration and adaptation. The aim is to achieve accurate recognition, semantic description, and intelligent data platform adaptation of cattle behavior. By deeply integrating computer vision and natural language processing technologies, and based on a deep learning image description model, this invention achieves, for the first time, a direct mapping from cattle behavior images to natural language descriptions containing environmental information. This effectively solves the problem that existing methods can only identify the behavior itself but cannot express its relationship to the farming context.

[0047] Figure 2 A flowchart illustrating another method for cow behavior recognition and interpretation based on deep learning image description provided in this embodiment is shown below. Figure 2 As shown, the method includes the following steps: 2.1 Creation of a Cattle Behavior Image Description Dataset (Focusing on the Specific Needs of Cattle Farming Scenarios) (1) Collection of cattle behavior images: In real cattle farming scenarios, high-definition cameras are used to capture videos and images of cattle in various behavioral states such as standing, walking, lying down, looking back at their abdomen, eating, and drinking in multiple areas such as pens, pastures, drinking areas, and feeding areas. The focus is on collecting images that can reflect the physiological characteristics of cattle, covering the behavioral characteristics of different lighting conditions (daytime and evening), different stocking densities, and different cattle breeds (dairy cows and beef cattle). A total of no less than 2,000 valid original images are collected.

[0048] (2) Dataset Creation (Cattle Behavior Description Annotation): Develop specific language description rules applicable to cattle behavior description, requiring each image to simultaneously contain the following three core information aspects: cattle behavioral state characteristics, including action type, posture details, such as head down slowly eating, lying on the side with limbs extended; environmental characteristics, including the breeding scene, surrounding facilities, such as next to the feeding trough in the feeding area, inside the pasture fence; spatial relationships, including the positional relationship between the cattle and the facility environment, such as the head near the feeding trough, the body located in the center of the pasture. Perform natural language description annotation on each original image according to the rules to form a structured, scenario-based cattle behavior image description dataset, for example: a cow head down eating grass in the pasture, two cows lying down resting in the pen, a cow standing and drinking water next to the water trough, etc.

[0049] 2.2 Construction and Training of Deep Learning Image Description Model (Combined with Specific Optimization for Cattle Behavior Recognition) (1) Training of deep learning image description model (based on CNN+RNN+Transformer architecture): The CNN-RNN encoder-decoder framework is used as the basic model, such as Figure 3 As shown, CNN is responsible for extracting visual features from images of cattle behavior, while RNN is responsible for generating corresponding natural language descriptive text.

[0050] On the encoder side, to address the needs of cattle behavior recognition, an improved CBAM attention module is embedded after the 2nd-4th stage convolutional layers of ResNet50 to improve the accuracy of cattle behavior feature extraction. Specifically, the channel attention module emphasizes the feature channel weights of key body parts such as the limbs, head, and torso, while suppressing background interference such as pen pillars and pasture weeds. The spatial attention module focuses on key areas of cattle behavior, such as the mouth when grazing, the junction of the head and water trough when drinking, and the hooves when walking.

[0051] On the decoder side, when introducing an attention mechanism based on RNN, the attention weight allocation strategy is optimized to address the temporal correlation of cattle behavior, such as the continuous movements of the limbs when walking, and the process of lowering the head, chewing, and raising the head when feeding. This enables the model to accurately match the temporal logic of the behavior when generating text descriptions, and to focus on key information related to the behavior and environment in the image.

[0052] The completed cattle behavior image description dataset was divided into training, validation, and test sets in a 7:2:1 ratio for model training and debugging. During training, model parameters were optimized, and hyperparameters such as learning rate and batch size were adjusted in real-time using the validation set to ensure the model's description accuracy on the test set was no less than 85%, and the processing time per image was ≤0.5 seconds, meeting the real-time monitoring needs of pastures.

[0053] (2) Deep learning image description model test (based on CNN+RNN+Transformer architecture): Use the test set to evaluate the performance of the trained model to ensure that its description accuracy and real-time performance meet the requirements of practical applications.

[0054] 2.3 Semantic Description and Interpretation of Cattle Behavior (Adapting to the Needs of Livestock Management Decision-Making) (1) Semantic description of bovine behavior (natural language description of behavior), i.e. Figure 4 Semantic word segmentation and keyword extraction: Natural language processing technology is used to segment the natural language description text of cattle behavior generated by the model, focusing on extracting key information such as cattle-specific behavioral keywords (feeding, looking back at the abdomen, lameness, etc.), cattle breed keywords (dairy cows, beef cows, etc.), scene keywords (pen, pasture, water trough, etc.), and location keywords (side, middle, inside, etc.), while ignoring irrelevant and redundant words.

[0055] (2) Bovine behavioral feature interpretation algorithm processing (extraction of key behavioral parameters), i.e. Figure 4 Behavioral feature mapping rule formulation: Based on the data storage specifications and breeding management needs of the intelligent breeding data platform, mapping rules between semantic keywords and cattle behavioral feature data are formulated to ensure that the mapping results directly serve breeding decisions. For example, mapping dairy cows eating next to the feeding trough in the feeding area is as follows: Behavior type: eating; Animal breed: dairy cow; Scene type: feeding area trough; Location code: next to; Behavioral status: normal.

[0056] (3) Structured data generation (i.e.) Figure 4 (Structured behavioral data generation in the process): Based on the mapping rules, the extracted keywords are converted into structured cattle behavioral data. The data format includes behavior codes, animal breeds, scene codes, location codes, status codes, timestamps, etc., to ensure that the data meets the input requirements of the intelligent breeding data platform.

[0057] 2.4 Integration with the intelligent aquaculture data platform (data storage / analysis / application) The generated structured cattle behavior data is then standardized in format (i.e. Figure 4 The data format is standardized, and the data is encapsulated in JSON format. Through the interface adaptation module, it can be seamlessly connected with the grassland animal husbandry traceability big data platform and the existing intelligent management system of the ranch, supporting real-time data upload, storage and query.

[0058] This invention provides a method for cattle behavior recognition and interpretation based on deep learning image description. This method can simultaneously identify cattle behaviors and their association with the surrounding farming environment, achieving an overall recognition accuracy of no less than 85%. It can effectively distinguish similar behaviors in different scenarios, providing a more comprehensive and accurate basis for farming decisions. Simultaneously, by describing cattle behavior in natural language, farming managers can quickly grasp the real-time status of cattle without needing a background in computer vision or deep learning, significantly lowering the barrier to entry and improving management decision-making efficiency. Through a dedicated semantic interpretation algorithm and data standardization processing, seamless integration between the recognition results and the intelligent farming data platform is achieved, effectively solving the last-mile problem in data flow and application in existing technologies and improving data utilization efficiency. Based on accurate and scenario-based behavior recognition and analysis, it can provide early warnings of cattle health abnormalities and estrus status 3-4 days in advance, helping to improve disease cure rates and reproductive success rates, reduce overall farming costs, and reduce manual inspection workload by more than 30%. The provided dedicated dataset construction method and image description model framework incorporating attention mechanisms can be transferred and applied to behavior recognition of other livestock such as sheep and horses, providing a reusable technical solution for the overall digital and intelligent transformation and upgrading of the livestock industry.

[0059] It should be noted that, in the embodiments of the present invention, if the above-described method for recognizing and interpreting cattle behavior based on deep learning image description is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of the present invention are not limited to any specific hardware and software combination.

[0060] Correspondingly, embodiments of the present invention provide an electronic device, Figure 5 This is a schematic diagram of the composition structure of the electronic device provided in the embodiments of the present invention, such as... Figure 5As shown, the electronic device 500 includes at least a processor 501 and a computer-readable storage medium 502 configured to store executable instructions, wherein the processor 501 generally controls the overall operation of the electronic device. The computer-readable storage medium 502 is configured to store instructions and applications executable by the processor 501, and may also cache data to be processed or processed by various modules in the processor 501 and the electronic device 500, which may be implemented using flash memory or random access memory (RAM).

[0061] This invention provides a storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this invention, for example... Figure 1 The method shown.

[0062] In some embodiments, the storage medium may be a computer-readable storage medium, such as a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disc, or a compact disk-read-only memory (CD-ROM); or it may be a device that includes one or any combination of the above-mentioned memories.

[0063] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0064] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file containing other programs or data, for example, in one or more scripts within a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0065] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of the present invention are included within the scope of protection of the present invention.

[0066] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the invention, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the invention. The sequence numbers of the above-described embodiments of the invention are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0067] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not performed.

[0068] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for recognizing and interpreting cattle behavior based on deep learning image description, characterized in that, The method includes: Multiple original images from cattle farming scenarios are acquired, and each original image is annotated with natural language descriptions to form a cattle behavior image description dataset. The cow behavior image to be identified is input into a trained cow behavior image model to extract the corresponding natural language description text; the trained cow behavior image model is trained based on the cow behavior image description dataset; The natural semantic description text is segmented to extract keywords related to cattle behavior, breed, scene, and location. According to the preset behavior-environment-space cattle-specific mapping rules, the extracted keywords are converted into structured cattle behavior data, which includes behavior codes, breed codes, scene codes, location codes, status codes and timestamps. The structured cattle behavior data is standardized and encapsulated into JSON data format, then sent to the intelligent livestock farming data platform through the interface adaptation module to complete the data connection.

2. The method according to claim 1, characterized in that, The trained image model of cattle behavior is trained through the following steps: The CNN-RNN encoder-decoder framework is used as the initial model, where the CNN is used to extract visual features of cattle behavior images and the RNN is used to generate corresponding natural language description text. At the encoder end, an improved CBAM attention module is embedded after the second to fourth stage convolutional layers of the ResNet50 network structure; Among them, the improved CBAM attention module uses the attention module to strengthen the feature channel weights of key body parts of cattle, and the spatial attention module is used to focus on key areas of cattle behavior. On the decoder side, when introducing an attention mechanism based on RNN, the attention weight allocation strategy is optimized for the temporal correlation of cow behavior; The cattle behavior image description dataset is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The initial model is trained and its parameters are optimized using the training set and the validation set to obtain the trained cattle behavior image model.

3. The method according to claim 1, characterized in that, The cattle farming scene includes at least one of the following: pen, pasture, watering area, and feeding area; the number of original images is no less than 2,000, and the original images cover different lighting conditions, stocking densities, and different breeds of cattle.

4. The method according to claim 1, characterized in that, When performing natural language description annotation on each of the original images, the annotation content includes the characteristics of cattle behavior, the characteristics of the breeding environment, and the spatial relationship between the cattle and the facility environment; The behavioral characteristics of the cattle include action type and posture details; The characteristics of the aquaculture environment include the aquaculture setting and surrounding facilities; The spatial relationships include the positional relationship between the cattle and the facility environment.

5. The method according to claim 2, characterized in that, The key areas of cattle behavior include the mouth when feeding, the junction of the head and the water trough when drinking, and the hooves when walking. The temporal correlation of the cattle behavior includes the continuous movements of the limbs when walking, and the process of lowering the head, chewing, and raising the head when feeding.

6. A system for recognizing and interpreting bovine behavior based on deep learning image description, characterized in that, The system includes: The acquisition module is used to acquire multiple original images in the cattle breeding scenario and to perform natural language description annotation on each of the original images to form a cattle behavior image description dataset. The extraction module is used to input the cow behavior image to be identified into the trained cow behavior image model and extract the corresponding natural semantic description text; the trained cow behavior image model is trained based on the cow behavior image description dataset; The word segmentation module is used to segment the natural semantic description text and extract keywords related to cattle behavior, breed, scene, and location. The conversion module is used to convert the extracted keywords into structured cattle behavior data according to the preset behavior-environment-space cattle-specific mapping rules. The structured cattle behavior data includes behavior codes, breed codes, scene codes, location codes, status codes and timestamps. The standardization module is used to standardize the structured cattle behavior data and encapsulate it into JSON data format. The data is then sent to the intelligent livestock farming data platform through the interface adaptation module to complete the data connection.

7. An electronic device, characterized in that, include: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the method for recognizing and interpreting bovine behavior based on deep learning image description as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the cattle behavior recognition and interpretation method based on deep learning image description as described in any one of claims 1 to 5.

Images