Multi-mode fusion livestock and poultry house manure cleaning method, system and equipment

Abstract

The invention discloses a multi-modal fusion livestock and poultry house manure cleaning method, system and equipment, and relates to the field of breeding cleaning, the method comprises the following steps: collecting and identifying a livestock and poultry house scene image, and if the livestock and poultry house scene image is identified as a slatted floor, executing a slatted floor manure cleaning strategy: obtaining a manure image and planning an optimal path, lifting the scraper to drive the rolling brush to sweep and extrude along a path, detecting the ammonia concentration to determine whether deodorization is completed, and if not, circularly executing; after a scraper blade is lowered, a driving wheel moves along the ditch, a spray head is controlled to wash after the position and the form of excrement are recognized through images, the ammonia concentration is detected when the end point is reached to determine whether deodorization is conducted or not, and finally the starting point is returned. According to the invention, multi-mode manure cleaning is carried out based on visual perception and ammonia gas concentration, the integrated full-automatic intelligent manure cleaning process of scene recognition, path planning, manure cleaning and active deodorization of the livestock and poultry house is realized, the accuracy and cleaning efficiency of manure cleaning are improved, and the error rate is reduced.

Description

Technical Field

[0001] This application relates to the field of livestock cleaning, and in particular to a multimodal integrated method, system and equipment for cleaning livestock and poultry manure. Background Technology

[0002] Livestock and poultry farming is a crucial link in ensuring human food supply. However, the disposal of livestock and poultry manure has always been a key problem for farmers. Manure cleaning mainly relies on manual labor using brooms, shovels, and other tools, or transportation using handcarts. This method is slow, labor-intensive, and unsuitable for the cleaning needs of large-scale farms. Furthermore, with the continuous rise in labor costs, the economic viability of manual cleaning is gradually decreasing.

[0003] Existing mobile fecal scraping robots rely on simple sensors (such as collision sensors and infrared sensors), which make it difficult to accurately identify the distribution of feces, and they are prone to collisions or missing cleaning areas. In addition, existing mobile fecal scraping robots cannot make accurate judgments and controls based on the actual situation, and cannot achieve fully automatic intelligent cleaning. They have poor cleaning effect, low efficiency, and high error rate. Summary of the Invention

[0004] The purpose of this application is to provide a multimodal fusion method, system and equipment for cleaning livestock and poultry manure, which can realize a fully automatic intelligent manure cleaning process integrating scene recognition, path planning, manure cleaning and active deodorization in livestock and poultry houses, improve the accuracy and cleaning efficiency of manure cleaning in livestock and poultry houses and reduce the error rate.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] In a first aspect, this application provides a multimodal fusion method for cleaning manure in livestock and poultry houses. This method is applied to a manure cleaning device, which includes a scraper mechanism, a nozzle mechanism, a roller brush mechanism, and a drive wheel. The method includes: acquiring a scene image of a target livestock and poultry house; performing scene recognition on the scene image; controlling the manure cleaning device to execute a slatted floor cleaning strategy when the recognition result is a slatted floor; and controlling the manure cleaning device to execute a manure ditch cleaning strategy when the recognition result is a manure ditch. The slatted floor cleaning strategy includes: acquiring an image of manure on the slatted floor; planning a cleaning path based on the manure image to obtain an optimal path; controlling the scraper to rise and controlling the drive wheel to drive the roller brush along the optimal path to sweep and compress the manure; acquiring the ammonia concentration and determining whether the ammonia concentration exceeds [a certain threshold]. If the concentration threshold is met, the spray nozzle mechanism is controlled to deodorize; otherwise, proceed to the next step. It is determined whether all manure on the optimal path has been cleaned. If so, the slatted floor cleaning is complete; otherwise, return to "Acquire image of manure on the slatted floor of the target livestock shed". The manure ditch cleaning strategy includes: controlling the scraper to lower; controlling the drive wheel to move along the manure ditch and acquiring an image of the manure surface; identifying manure based on the surface image to obtain the location and morphology of the manure, and controlling the spray nozzle mechanism to rinse the manure based on the location and morphology; when the drive wheel reaches the end of the manure ditch, the ammonia concentration is acquired, and it is determined whether the ammonia concentration exceeds the ammonia concentration threshold. If so, the drive wheel is controlled to drive the spray nozzle mechanism to deodorize; otherwise, the drive wheel is controlled to return to the beginning of the manure ditch, and the slatted floor cleaning is complete.

[0007] Secondly, a multimodal fusion manure cleaning system for livestock and poultry houses includes: a depth camera, an ammonia concentration analyzer, a manure cleaning device, and a strategy generation module; the manure cleaning device includes: a scraper mechanism, a nozzle mechanism, a roller brush mechanism, and a drive wheel; the strategy generation module is connected to the depth camera and the ammonia concentration analyzer respectively; the manure cleaning device is connected to the strategy generation module; the strategy generation module is used to obtain a slatted floor manure cleaning strategy or a manure ditch manure cleaning strategy using the multimodal fusion manure cleaning method described above, and to control the manure cleaning device to clean manure based on the strategy.

[0008] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multimodal fusion livestock and poultry house manure cleaning method described above.

[0009] Based on the specific embodiments provided in this application, the following technical effects are disclosed.

[0010] This application first acquires and identifies images of the livestock and poultry shed scene. If the shed is identified as having a slatted floor, a slatted floor cleaning strategy is implemented: acquiring images of the manure and planning the optimal path, raising the scraper and driving the roller brush to sweep and compress along the path, while simultaneously detecting ammonia concentration to determine whether deodorization is needed; if not completed, the process is repeated. If the shed is identified as having a manure ditch, a manure ditch cleaning strategy is implemented: lowering the scraper and driving the wheels to move along the ditch, controlling the nozzles to rinse based on the image recognition of the manure location and shape, detecting ammonia concentration at the endpoint to determine whether deodorization is needed, and finally returning to the starting point. This application achieves a fully automated intelligent manure cleaning process integrating scene recognition, path planning, manure cleaning, and active deodorization in livestock and poultry sheds through multimodal manure cleaning based on visual perception and ammonia concentration. This significantly improves the accuracy and efficiency of manure cleaning in livestock and poultry sheds, while reducing the risk of human intervention and operational errors. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a multimodal fusion method for cleaning livestock and poultry manure in an embodiment of this application. Figure 1 .

[0013] Figure 2 A flowchart illustrating a multimodal fusion method for cleaning livestock and poultry manure in an embodiment of this application. Figure 2 .

[0014] Figure 3 A schematic diagram of a multimodal fusion manure removal system for livestock and poultry houses provided in this application embodiment. Figure 1 .

[0015] Figure 4 A schematic diagram of a multimodal fusion manure removal system for livestock and poultry houses provided in this application embodiment. Figure 2 .

[0016] Figure 5 This is a schematic diagram of the distribution of fecal matter in the slatted floor and manure ditch provided in an embodiment of this application.

[0017] Figure 6 This is a schematic diagram of the scene recognition flowchart provided in an embodiment of this application.

[0018] Figure 7 This is a visual illustration of slatted floor identification provided in an embodiment of this application; wherein, Figure 7 (a) in the image is the original image; Figure 7 (b) in the image is the preprocessed image; Figure 7 (c) in the figure represents the classification result.

[0019] Figure 8 This is a flowchart illustrating the fecal waste identification and location process provided in an embodiment of this application.

[0020] Figure 9 A schematic diagram of the structure of the lightweight YOLOv11n model provided in the embodiments of this application.

[0021] Figure 10 This is a schematic diagram of the path planning results provided in an embodiment of this application.

[0022] Figure 11 This is a schematic diagram of the iterative process provided in an embodiment of this application.

[0023] Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0024] Figure labels: Depth camera-1; Ammonia concentration analyzer-2; Strategy generation module-3; and manure removal device-4. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Example 1, as Figures 1-2 As shown, this embodiment provides a multimodal fusion manure cleaning method for livestock and poultry houses. The multimodal fusion manure cleaning method for livestock and poultry houses is applied to a manure cleaning device, which includes a scraper mechanism, a nozzle mechanism, a roller brush mechanism, and a drive wheel. The method includes the following steps.

[0028] S1. Acquire scene images of the target livestock and poultry shed.

[0029] S2. Perform scene recognition on the scene image of the target livestock and poultry house. When the recognition result is a slatted floor, control the manure cleaning device to execute the slatted floor manure cleaning strategy; when the recognition result is a manure ditch, control the manure cleaning device to execute the manure ditch manure cleaning strategy; wherein, Figure 5 The distribution of fecal matter in the slatted floor and manure ditch is shown.

[0030] Furthermore, such as Figures 6-7 As shown, step S2 specifically includes the following steps.

[0031] S21. Denoise the scene image and update the scene image.

[0032] S22. Convert the scene image to grayscale to obtain the grayscale scene image.

[0033] S23. Binarize the grayscale scene image to obtain the binarized scene image.

[0034] Furthermore, the binarization method used is the Otsu adaptive thresholding method.

[0035] S24. Use the gray-level co-occurrence matrix to perform texture analysis on the binarized scene image to obtain the target texture features.

[0036] S25. Based on the target texture features, the scene is identified. If the contrast of the target texture features is higher than the contrast threshold, the identification result is a slatted floor; otherwise, the identification result is a manure ditch.

[0037] S26. When the identification result is a slatted floor, control the manure cleaning device to execute the slatted floor manure cleaning strategy; when the identification result is a manure ditch, control the manure cleaning device to execute the manure ditch manure cleaning strategy.

[0038] in, Figure 7 (a) shows the original image; Figure 7 (b) shows the preprocessed image; Figure 7 (c) shows the classification results. Figure 7 The process of visualizing slatted floor identification is shown.

[0039] In practical applications, the scene recognition process is as follows: distinguishing between slatted floors and manure ditches is key to selecting the end effector for the manure removal system. Figure 6 This paper describes a scene recognition algorithm for slatted floors and manure ditches. First, scene images are acquired in real-time from a depth camera. In image preprocessing, median filtering and Gaussian filtering are applied to the input images to eliminate noise caused by the environment or hardware. Subsequently, the images are converted from RGB space to grayscale space to facilitate texture and color feature extraction. Binarization is introduced, and the grayscale image is segmented using the Otsu adaptive thresholding method to enhance the boundary features of the floor and manure ditches. In texture feature extraction, texture features such as contrast and uniformity are calculated from the preprocessed images using the gray-level co-occurrence matrix to extract the regular grid texture of the slatted floors and the texture features of the manure ditches. Simultaneously, local binary mode is combined to enhance the detailed texture description of the regions. In feature classification, using the obtained texture features, images with high contrast are identified as slatted floors, while images with low contrast and smooth surfaces correspond to manure ditches.

[0040] 1) The slatted floor manure removal strategy includes: acquiring images of manure on the slatted floor; planning the manure removal path based on the images to obtain the optimal path; controlling the scraper to rise and controlling the drive wheel to drive the roller brush to sweep and squeeze the manure along the optimal path; acquiring the ammonia concentration and determining whether the ammonia concentration exceeds the concentration threshold. If so, controlling the nozzle mechanism to deodorize; if not, proceeding to the next step; determining whether all manure on the optimal path has been cleaned. If so, the slatted floor manure removal is complete; if not, returning to "acquire images of manure on the slatted floor of the target livestock and poultry house".

[0041] Furthermore, based on the fecal soil image of the slatted floor, a fecal cleaning path is planned to obtain the optimal path. Specifically, this includes: using a lightweight Yolov11n network to identify fecal soil in the RGB image of the slatted floor fecal soil image to obtain the two-dimensional location coordinates of the fecal soil; based on the depth image of the slatted floor fecal soil image, using the camera intrinsic parameter matrix to convert the two-dimensional location coordinates of the fecal soil into three-dimensional location coordinates of the fecal soil; and performing path planning based on the three-dimensional location coordinates of the fecal soil to obtain the optimal path.

[0042] Furthermore, the path planning algorithm used is a genetic algorithm.

[0043] In practical applications, the process of applying fecal waste identification and location algorithms is as follows.

[0044] This application combines lightweight Yolov11n and a RealSense depth camera for manure recognition. The depth camera acquires 640×480 RGB images and 640×480 depth images from the livestock environment. It then identifies manure and outputs image coordinates including the manure. Based on the image coordinates output by the network, and combined with the depth values ​​of the depth images, the image coordinates are converted into three-dimensional spatial coordinates using the camera intrinsic parameter matrix. The process is as follows: Figure 8 As shown.

[0045] Among them, such as Figure 9As shown, the lightweight Yolov11n network structure is as follows: Layer 0: Conv, input is the RGB image captured by the depth camera, convolution operation is performed, kernel size is 3x3, stride is 2, padding is 1. Through this operation, the feature map size is halved, and the number of output channels is 64. Layer 1: Conv, input is the 64-channel feature map from layer 0, convolution operation is performed, kernel size is 3x3, stride is 2, padding is 1. The number of output channels is 128. Layer 2: C3k2, input is the 128-channel feature map from layer 1, the number of output channels is 256. Layer 3: Conv, input is the 256-channel feature map from layer 2, convolution operation is performed, kernel size is 3x3, stride is 2, padding is 1. The number of output channels is 256. Layer 4: C3k2, input is the 256-channel feature map from layer 3, the number of output channels is 512. Layer 5: Conv, takes a 512-channel feature map from Layer 4 as input, performs a convolution operation with a 3x3 kernel, a stride of 2, and padding of 1. Outputs 512 channels. Layer 6: GhostModule, takes a 512-channel feature map from Layer 5 as input, uses a lightweight Ghost feature generation module, and outputs 512 channels. Layer 7: Conv, takes a 512-channel feature map from Layer 6 as input, performs a convolution operation with a 3x3 kernel, a stride of 2, and padding of 1. Outputs 1024 channels. Layer 8: C3k2, takes a 1024-channel feature map from Layer 7 as input, and outputs 1024 channels. Layer 9: SPPF, takes a 1024-channel feature map from Layer 8 as input, performs spatial pyramid pooling using a 5x5 pooling kernel to aggregate multi-scale features. Outputs 1024 channels. Layer 10: C2PSA, takes a 1024-channel feature map from layer 9 as input, uses the C2PSA module for attention-enhanced global context modeling, and outputs 1024 channels. Layer 11: Upsample, upsamples the 1024-channel feature map from layer 10 by a factor of 2, using nearest neighbor interpolation. Outputs 1024 channels. Layer 12: Concat, concatenates the feature map from layer 11 with the feature map from layer 6, and outputs 1536 channels. Layer 13: C3k2, takes a 1536-channel feature map from layer 12 as input, and outputs 512 channels. Layer 14: Upsample, upsamples the 512-channel feature map from layer 13 by a factor of 2, using nearest neighbor interpolation. Output feature map size is P3 level. Layer 15: Concat, concatenates the feature map from layer 14 with the feature map from layer 4, and outputs 1024 channels. Layer 16: C3k2, takes a 1024-channel feature map from layer 15 as input, and outputs 256 channels. Layer 17: Conv, takes a 256-channel feature map from layer 16 as input, performs a convolution operation, with a kernel size of 3x3, a stride of 2, and padding of 1.The output channel count is 256. Layer 18: Concat, connecting the feature maps from layer 17 and layer 13, outputting 768 channels. Layer 19: C3k2, taking the 768-channel feature map from layer 18 as input, outputting 512 channels. Layer 20: Conv, taking the 512-channel feature map from layer 19 as input, performing a convolution operation with a 3x3 kernel, a stride of 2, and padding of 1. Outputting 512 channels. Layer 21: Concat, connecting the feature maps from layer 20 and layer 10, outputting 1536 channels. Layer 22: C3k2, taking the 1536-channel feature map from layer 21 as input, outputting 1024 channels. Layer 23: Detect, combining the detection feature maps from layers 16, 19, and 22 at different scales for detecting small, medium, and large targets respectively. The output includes the target bounding box and two-dimensional coordinates.

[0046] The process of coordinate transformation from two-dimensional to three-dimensional is as follows.

[0047] Step 1: After obtaining the 2D coordinates (u,v) from the RGB image through object detection, read the depth value corresponding to the target pixel (u,v) from the depth map. Z and the camera's intrinsic parameter matrix K .

[0048] .

[0049] in: f x ，f y It's the focal length. c x ,c y ) is the center of the optical axis.

[0050] Step 2: Use the camera intrinsic parameter matrix to convert the pixel coordinates into normalized image plane coordinates. X norm , Y norm ).

[0051] .

[0052] Step 3: Transform to the 3D camera coordinate system, based on the depth value. Z Calculate the three-dimensional coordinates in the camera coordinate system. X , Y , Z ).

[0053] .

[0054] The process of optimal path planning is as follows.

[0055] In the cleaning area of ​​a slatted floor, the optimal path planning problem for the manure removal system can be viewed as a traveling salesman problem. In this problem, the manure removal system needs to traverse multiple manure points and return to the starting point to plan the shortest path.

[0056] The process and formula for solving this problem based on genetic algorithms are as follows.

[0057] The first step is to define the distribution area of ​​fecal waste as a node set { P 0 ,P 1 ,P 2 ,…，P n Each node corresponds to a sewage cleaning point. Randomly generated. M Path { T 0 ,T 1 ,T 2 ,…，T m Each path represents an individual, and the path is an arrangement of manure cleaning points. The total length of each path is calculated.

[0058] .

[0059] .

[0060] in, fecal matter Two-dimensional coordinates.

[0061] The second step is to calculate the fitness of each path, where =0.0001, its function is to avoid division by zero, fitness The calculation formula is as follows.

[0062] .

[0063] The third step is to select based on fitness. The path is used as the parent, and the selection probability is... P i The calculation formula is as follows.

[0064] .

[0065] Cumulative probability The calculation formula is as follows, where ; .

[0066] .

[0067] No. i The cumulative interval of each path is [ C i+1 ,C i Generate a random number in the interval [0, 1). r , confirmed r The cumulative probability interval to which it belongs, if That is, the individual T i The chosen parent generation.

[0068] The fourth step is the mutation operation, which randomly selects two nodes to swap positions. and Generate new offspring .

[0069] .

[0070] Calculate the fitness function of offspring and select based on fitness.

[0071] Fifth, repeat steps three and four until the maximum number of iterations is reached, then output the optimal path. Among these steps... Figures 10-11 The path planning results (optimal path) and iterative process are shown.

[0072] 2) The manure cleaning strategy for the slatted floor includes: controlling the scraper to lower; controlling the drive wheel to move along the manure ditch and acquiring images of the manure surface; identifying the manure based on the images to obtain the location and morphology of the manure, and controlling the nozzle mechanism to flush the manure based on the location and morphology of the manure; when the drive wheel reaches the end of the manure ditch, acquiring the ammonia concentration and determining whether the ammonia concentration exceeds the ammonia concentration threshold. If so, controlling the drive wheel to drive the nozzle mechanism to deodorize; if not, controlling the drive wheel to return to the beginning of the manure ditch, and the cleaning of the slatted floor is completed.

[0073] Furthermore, fecal contamination is identified based on surface fecal contamination images to obtain the location and morphology of the fecal contamination. The nozzle mechanism is then controlled to flush the fecal contamination based on this location and morphology. Specifically, this includes: using a lightweight Yolov11n network to identify fecal contamination from the RGB images of the surface fecal contamination, obtaining two-dimensional location coordinates; using the depth image of the fecal contamination from the slatted floor fecal contamination image, converting the two-dimensional location coordinates into three-dimensional location coordinates using a camera intrinsic parameter matrix, and identifying the fecal contamination morphology; using the three-dimensional location coordinates as the fecal contamination location, and controlling the nozzle mechanism to flush the fecal contamination based on the location and morphology.

[0074] Furthermore, the specific forms of fecal matter include: attachment height, coverage area, and shape.

[0075] Furthermore, the concentration threshold includes at least 25 ppm.

[0076] Optionally, the control of the nozzle mechanism includes: water flow intensity, spray angle, and rinsing time.

[0077] In practical applications, this application focuses on slatted floors, with data collection emphasizing the identification of fecal distribution to enable the cleaning system to plan the optimal cleaning route. For the fecal ditch area, the morphological characteristics such as the area of ​​fecal coverage on the ground, the adhesion height of fecal on the ditch walls, and the coverage area are crucial for accurately controlling the water flow intensity, spray angle, and rinsing time of the nozzles. This effectively avoids excessive rinsing that wastes water resources and ensures thorough cleaning of the walls.

[0078] Example 2, as Figures 3-4 As shown, a multimodal fusion livestock and poultry house manure cleaning system includes: a depth camera 1, an ammonia concentration analyzer 2, a manure cleaning device 4, and a strategy generation module 3; the manure cleaning device 4 includes: a scraper mechanism, a nozzle mechanism, a roller brush mechanism, and a drive wheel.

[0079] The strategy generation module 3 is connected to the depth camera 1 and the ammonia concentration analyzer 2 respectively; the manure cleaning device 4 is connected to the strategy generation module 3; the strategy generation module 3 is used to obtain the slatted floor manure cleaning strategy or the manure ditch manure cleaning strategy by applying the above multimodal fusion manure cleaning method, and control the manure cleaning device 4 to clean the manure based on the strategy.

[0080] Optionally, the scraper mechanism, nozzle mechanism, roller brush mechanism, and drive wheel are all connected to the strategy generation module 3.

[0081] Optionally, the multimodal fusion livestock and poultry house manure removal system also includes a power module for powering the depth camera 1, ammonia concentration analyzer 2, manure removal device 4, and strategy generation module 3.

[0082] Optionally, the strategy generation module 3 specifically includes: an upper-level controller (Jetson Nano) and a lower-level controller (STM32F103C8T6); the input of the upper-level controller is connected to the depth camera 1 and the ammonia concentration analyzer 2 respectively, and the output of the upper-level controller is connected to the lower-level controller; the upper-level controller is used to perform scene recognition on the scene image of the target livestock and poultry house, and when the recognition result is a slatted floor, a slatted floor manure cleaning strategy is generated; when the recognition result is a manure ditch, a manure ditch manure cleaning strategy is generated.

[0083] The bottom-level controller is connected to the manure removal device 4; the bottom-level controller is used to: control the manure removal device 4 to execute the manure removal strategy when the identification result is a slatted floor; and control the manure removal device 4 to execute the manure removal strategy when the identification result is a manure ditch.

[0084] The upper-level controller connects to depth camera 1 (RealSense D435i) and ammonia concentration analyzer 2 (TDLAS) via USB interface. It processes visual data, ammonia concentration data, and performs path planning, and communicates with the lower-level controller via GPIO interface. The lower-level controller directly controls multiple actuators via GPIO interface: a lifting motor drives the scraper mechanism for vertical movement, a servo motor controls the direction of the nozzle mechanism, a water pump drives the nozzle mechanism for flushing / deodorizing modes, a roller brush motor drives the roller brush mechanism for sweeping, and a drive motor drives the overall movement of the manure removal system. The upper-level controller sends motion and cleaning commands to the lower-level controller, which then drives the corresponding actuators to complete the manure removal operation and reports the execution status back to the upper-level controller via GPIO interface. Environmental data collected by depth camera 1 and ammonia concentration analyzer 2 is directly used by the upper-level controller for intelligent decision-making, forming a closed-loop control system.

[0085] In practical application, this application first identifies the current area of ​​the (multimodal fusion livestock and poultry house manure cleaning system) to determine whether it is a slatted floor or a manure ditch. If it is a slatted floor area, the manure cleaning system will identify the manure, plan a cleaning path, and mainly use roller brushes for sweeping and squeezing. If the ammonia concentration in the current area exceeds the standard, the nozzle deodorization mode will be activated. After the current area is cleaned, it will move to the next cleaning area until all areas are cleaned. If it is a manure ditch area, the manure cleaning system starts from one end of the ditch and moves along the ditch to the other end. During the process, it simultaneously identifies and cleans the manure on the walls and the ground. First, the nozzles are activated to rinse the manure on the walls, and at the same time, the scrapers are activated to clean the manure on the ground. The cleaning status is continuously judged. If it is not completed, it continues to move and clean until it is confirmed that the manure has been cleaned. Then, the nozzle deodorization mode is activated, and it returns to the starting point to complete the entire cleaning process. This application utilizes an ammonia concentration analyzer to monitor ambient ammonia levels in real time, combines this with machine vision to accurately identify manure distribution, and makes intelligent decisions based on multimodal perception data. It dynamically adjusts parameters such as roller brush speed, scraper height, and nozzle pressure to achieve adaptive and refined control of livestock manure cleaning. This system can automatically select the optimal working mode according to environmental conditions, significantly improving cleaning efficiency and environmental adaptability.

[0086] Example 3: This application also provides a computer device, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 12As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores and processes data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements the methods described above.

[0087] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multimodal fusion method for cleaning livestock and poultry manure, wherein the multimodal fusion method is applied to a manure cleaning device, the manure cleaning device comprising: The scraper mechanism, nozzle mechanism, roller brush mechanism, and drive wheel are characterized in that the method includes: Acquire scene images of the target livestock and poultry shed; Scene recognition is performed on the scene image of the target livestock and poultry house. When the recognition result is a slatted floor, the manure cleaning device is controlled to execute the slatted floor manure cleaning strategy; when the recognition result is a manure ditch, the manure cleaning device is controlled to execute the manure ditch manure cleaning strategy. The slatted floor manure removal strategy includes: acquiring images of manure on the slatted floor; planning a manure removal path based on the images to obtain an optimal path; controlling the scraper to rise and controlling the drive wheel to drive the roller brush along the optimal path to sweep and squeeze the manure; acquiring the ammonia concentration and determining whether the ammonia concentration exceeds the concentration threshold; if so, controlling the nozzle mechanism to deodorize; if not, proceeding to the next step; determining whether all manure on the optimal path has been cleaned; if so, the slatted floor manure removal is complete; if not, returning to acquire images of manure on the slatted floor of the target livestock shed. The manure cleaning strategy includes: controlling the scraper to lower; controlling the drive wheel to move along the manure ditch and acquiring images of the manure surface; identifying the manure based on the surface images to obtain the location and shape of the manure, and controlling the nozzle mechanism to flush the manure based on the location and shape of the manure; when the drive wheel reaches the end of the manure ditch, acquiring the ammonia concentration and determining whether the ammonia concentration exceeds the ammonia concentration threshold; if so, controlling the drive wheel to drive the nozzle mechanism to deodorize; if not, controlling the drive wheel to return to the beginning of the manure ditch, and the cleaning of the slatted floor is completed.

2. The multimodal fusion manure removal method for livestock and poultry houses according to claim 1, characterized in that, Scene recognition is performed on the scene image of the target livestock and poultry house. When the recognition result is a slatted floor, the manure removal device is controlled to execute the slatted floor manure removal strategy. When the identification result is a manure ditch, the manure cleaning device is controlled to execute a manure ditch cleaning strategy, specifically including: Denoise the scene image and update the scene image; The scene image is converted to grayscale to obtain the grayscale scene image; The grayscale scene image is binarized to obtain the binarized scene image; Texture analysis of the binarized scene image is performed using the gray-level co-occurrence matrix to obtain the target texture features; Scene recognition is performed based on target texture features. If the contrast of the target texture features is higher than the contrast threshold, the recognition result is a slatted floor; otherwise, the recognition result is a manure ditch. When the identification result is a slatted floor, the manure removal device is controlled to execute the slatted floor manure removal strategy; when the identification result is a manure ditch, the manure removal device is controlled to execute the manure ditch manure removal strategy.

3. The multimodal fusion manure removal method for livestock and poultry houses according to claim 2, characterized in that, The binarization method used is the Otsu adaptive thresholding method.

4. The multimodal fusion manure removal method for livestock and poultry houses according to claim 1, characterized in that, Based on the image of fecal waste in slatted floors, a fecal cleaning path is planned to obtain the optimal path, which specifically includes: A lightweight Yolov11n network is used to identify fecal contamination in RGB images of slatted floor feces, obtaining two-dimensional location coordinates of the feces. The lightweight Yolov11n network adds a lightweight Ghost feature generation module to the Yolov11n network. Based on the depth image of fecal contamination from the slatted floor, the two-dimensional location coordinates of fecal contamination are converted into three-dimensional location coordinates using the camera intrinsic parameter matrix. The optimal path is obtained by path planning based on the three-dimensional location coordinates of feces and sewage.

5. The multimodal fusion manure removal method for livestock and poultry houses according to claim 1, characterized in that, Fecal waste is identified based on surface images to determine its location and morphology. The nozzle mechanism is then controlled to flush the waste based on this information. Specifically, this includes: A lightweight Yolov11n network is used to identify fecal contamination in RGB images of surface fecal contamination, obtaining two-dimensional location coordinates of the fecal contamination. The lightweight Yolov11n network adds a lightweight Ghost feature generation module to the Yolov11n network. Based on the depth image of fecal soil in slatted floors, the two-dimensional position coordinates of fecal soil are converted into three-dimensional position coordinates of fecal soil using the camera intrinsic parameter matrix, and the morphology of fecal soil is identified. The three-dimensional coordinates of the feces are used as the location of the feces, and the nozzle mechanism is controlled to flush the feces based on the location and shape of the feces.

6. The multimodal fusion method for cleaning livestock and poultry manure according to claim 1, characterized in that, The concentration threshold includes at least 25 ppm.

7. The multimodal fusion manure removal method for livestock and poultry houses according to claim 1, characterized in that, The specific forms of fecal matter include: attachment height, coverage area, and shape.

8. The multimodal fusion manure removal method for livestock and poultry houses according to claim 1, characterized in that, The path planning algorithm used is a genetic algorithm.

9. A multimodal fusion manure removal system for livestock and poultry houses, characterized in that, The multimodal fusion livestock and poultry house manure removal system includes: a depth camera, an ammonia concentration analyzer, a manure removal device, and a strategy generation module; the manure removal device includes: a scraper mechanism, a nozzle mechanism, a roller brush mechanism, and drive wheels; The strategy generation module is connected to the depth camera and the ammonia concentration analyzer respectively; the manure cleaning device is connected to the strategy generation module; the strategy generation module is used to obtain a slatted floor manure cleaning strategy or a manure ditch manure cleaning strategy by applying the multimodal fusion manure cleaning method of any one of claims 1-8, and control the manure cleaning device to clean manure based on the strategy.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the multimodal fusion manure removal method for livestock and poultry houses according to any one of claims 1-8.

Images