Method and device for feeding livestock and poultry by large-scale robot based on machine vision

Abstract

The present application relates to livestock intelligent breeding technical field, especially a kind of large-scale mechanical arm livestock feeding method and device based on machine vision, wherein, method includes: using the current depth image of livestock feed box that image acquisition equipment on transfer mechanical arm is collected, and intercepts region of interest in current depth image, to determine the bottom reference plane of feed box;According to bottom reference plane and current depth image, the amount of feed that needs to be loaded into transfer mechanical arm is calculated;The transfer mechanical arm of loading feed is smoothed to detection prepose, and the three-dimensional point cloud of livestock cage is captured by image acquisition equipment, to calculate the minimum bounding box center coordinates of feed box;According to the minimum bounding box center coordinates of livestock feed box and detection prepose, calculate spatial straight line trajectory, to control the transfer mechanical arm of loading feed to move to the top of target livestock feed box according to spatial straight line trajectory and feed release. Therefore, it solves the problems that existing automation equipment is difficult to realize intelligent and individual feeding.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent livestock breeding, and particularly relates to a large-scale mechanical arm livestock feeding method and device based on machine vision. BACKGROUND

[0002] With the rapid development of large-scale breeding, higher requirements are put forward for the precision and efficiency of livestock feeding. The traditional livestock feeding method mainly relies on manual operation, which not only has a great labor intensity and consumes a large amount of human resources, but also is difficult to achieve accurate control of the feed intake of each livestock in the feeding process. Due to the limitations of manual operation, it is often difficult to accurately judge the actual needs of livestock, which may cause excessive feeding or insufficient feeding of feed, and further lead to waste of feed or nutritional imbalance of livestock. In addition, manual feeding is also difficult to ensure the timeliness and uniformity of feeding, which may affect the growth rate and health status of livestock.

[0003] Although the prior art has been improved to some extent, for example, by using automatic equipment for feeding, there are still some obvious shortcomings. For example, although some automatic equipment can reduce the labor intensity, the precision of feed feeding and the identification of individual needs of livestock still need to be improved. At the same time, some automatic equipment has limitations in adapting to different types and different growth stages of livestock, and it is difficult to achieve truly intelligent and personalized feeding. SUMMARY

[0004] The present application provides a large-scale mechanical arm livestock feeding method and device based on machine vision to solve the problem that the existing automatic equipment is difficult to achieve intelligent and personalized feeding.

[0005] The first aspect embodiment of the present application provides a large-scale mechanical arm livestock and poultry feeding method based on machine vision, comprising the following steps: collecting a current depth image of a target livestock and poultry feed box by using an image acquisition device pre-installed on a target transfer mechanical arm; intercepting a region of interest in the current depth image and extracting a plurality of data points in the region of interest to fit the plurality of data points to obtain a bottom reference plane of the target livestock and poultry feed box; calculating a remaining feed volume in the target livestock and poultry feed box according to the bottom reference plane and the current depth image; generating a feed supplement demand according to the remaining feed volume and a preset feeding amount standard to load a corresponding amount of feed into the transfer mechanical arm according to the feed supplement demand; smoothly moving the transfer mechanical arm loaded with feed to a detection pre-posed position and capturing a target livestock and poultry cage three-dimensional point cloud by the image acquisition device; collecting a feed box point cloud cluster in the target livestock and poultry cage three-dimensional point cloud and calculating a feed box minimum bounding box center coordinate according to the feed box point cloud cluster; calculating a spatial straight line trajectory according to the feed box minimum bounding box center coordinate and the detection pre-posed position to control the transfer mechanical arm loaded with feed to move to directly above the target livestock and poultry feed box according to the spatial straight line trajectory and release feed into the target livestock and poultry feed box.

[0006] Optionally, the current depth image of the target livestock and poultry feed box is collected by using the image acquisition device pre-installed on the target transfer mechanical arm, comprising:

[0007] The image acquisition device pre-installed on the target transfer mechanical arm is calibrated by a checkerboard to obtain a calibrated image acquisition device; and the current depth image of the target livestock and poultry feed box is collected by using the calibrated image acquisition device.

[0008] Optionally, the region of interest in the current depth image is intercepted, and a plurality of data points in the region of interest are extracted to fit the plurality of data points to obtain the bottom reference plane of the target livestock and poultry feed box, comprising:

[0009] An edge detection is performed on the current depth image to obtain a binary edge image of the target livestock and poultry feed box; a peak detection is performed on the binary edge image to determine a binary boundary image of the target livestock and poultry feed box; the region of interest in the binary boundary image is intercepted, and a plurality of data points in the region of interest are extracted to fit the plurality of data points to obtain the bottom reference plane of the target livestock and poultry feed box.

[0010] Optionally, the remaining feed volume in the target livestock and poultry feed box is calculated according to the bottom reference plane and the current depth image, comprising:

[0011] The current depth image is subjected to multi-frame sliding average filtering processing to obtain a smoothed depth image; and the residual volume in the target livestock and poultry feeding box is calculated according to the height of each pixel point and the bottom reference plane.

[0012] Optionally, the method further comprises:

[0013] The joint space interpolation algorithm and the RRT* algorithm are used to smoothly move the feeding transfer robot arm from the feeding position to the detection pre-pose; and the image acquisition device is controlled to emit coded stripes at a preset frequency to capture the three-dimensional point cloud of the target livestock and poultry cage.

[0014] Optionally, the method further comprises:

[0015] The three-dimensional point cloud of the target livestock and poultry cage is subjected to voxel down-sampling and statistical outlier filtering processing to obtain a processed three-dimensional point cloud of the target livestock and poultry cage; a support plane in the processed three-dimensional point cloud of the target livestock and poultry cage is segmented to extract the feeding box point cloud cluster; and principal component analysis processing is performed on the feeding box point cloud cluster to calculate the minimum bounding box center coordinates of the feeding box.

[0016] The second aspect of the present application provides a large-scale robot arm livestock and poultry feeding device based on machine vision, comprising: an acquisition module configured to acquire a current depth image of a target livestock and poultry feeding box by using an image acquisition device pre-installed on a target transfer robot arm; a fitting module configured to intercept a region of interest in the current depth image and extract a plurality of data points in the region of interest to fit the plurality of data points to obtain a bottom reference plane of the target livestock and poultry feeding box; a volume calculation module configured to calculate a residual volume in the target livestock and poultry feeding box according to the bottom reference plane and the current depth image; a demand generation module configured to generate a feeding demand according to the residual volume and a preset feeding amount standard, and load a corresponding amount of feed into the transfer robot arm according to the feeding demand; a capture module configured to smoothly move the feeding transfer robot arm to a detection pre-pose and capture a three-dimensional point cloud of a target livestock and poultry cage by using the image acquisition device; a coordinate calculation module configured to collect a feeding box point cloud cluster in the three-dimensional point cloud of the target livestock and poultry cage and calculate minimum bounding box center coordinates of the feeding box according to the feeding box point cloud cluster; and a feeding module configured to calculate a spatial straight line trajectory according to the minimum bounding box center coordinates of the feeding box and the detection pre-pose, control the feeding transfer robot arm to move to directly above the target livestock and poultry feeding box according to the spatial straight line trajectory, and release the feed into the target livestock and poultry feeding box.

[0017] The third aspect of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for livestock and poultry feeding by a large-scale mechanical arm based on machine vision as described in the above embodiments.

[0018] The fourth aspect of the present application provides a computer program product, wherein the computer program / instruction is executed by the processor to implement the method for livestock and poultry feeding by a large-scale mechanical arm based on machine vision as described above.

[0019] The fifth aspect of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the program is executed by the processor to implement the method for livestock and poultry feeding by a large-scale mechanical arm based on machine vision as described above.

[0020] The method and device for livestock and poultry feeding by a large-scale mechanical arm based on machine vision provided by the embodiments of the present application can accurately identify the remaining amount in the feed box of each livestock and poultry, thereby fully understanding the feed intake of each livestock and poultry, and accurately calculating and controlling the amount of feed for each feeding, effectively avoiding the problems of excessive feeding or insufficient feeding caused by inaccurate judgment in traditional manual feeding, significantly reducing feed waste, and ensuring the nutritional balance of livestock and poultry.

[0021] Additional aspects and advantages of the present application will be in part apparent and in part pointed out below in the description of the application. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A flowchart of the method for livestock and poultry feeding by a large-scale mechanical arm based on machine vision provided by the embodiments of the present application;

[0024] Figure 2 A specific execution schematic diagram of the method for livestock and poultry feeding by a large-scale mechanical arm based on machine vision provided by the embodiments of the present application;

[0025] Figure 3A block schematic diagram of a livestock and poultry feeding device based on a machine vision and a large-scale mechanical arm according to an embodiment of the present application is provided.

[0026] Figure 4 A structural schematic diagram of an electronic device according to an embodiment of the present application is provided.

[0027] Legend: 1-transport mechanical arm, 2-transport box, 3-image acquisition device, 4-livestock and poultry box, 5-livestock and poultry cage. DETAILED DESCRIPTION

[0028] Embodiments of the present application are described in detail below with reference to examples shown in the accompanying drawings, in which the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0029] A livestock and poultry feeding method and device based on a machine vision and a large-scale mechanical arm according to an embodiment of the present application are described below with reference to the accompanying drawings.

[0030] Figure 1 A flowchart of a livestock and poultry feeding method based on a machine vision and a large-scale mechanical arm according to an embodiment of the present application is provided.

[0031] As shown in Figure 1 , the livestock and poultry feeding method based on a machine vision and a large-scale mechanical arm includes the following steps:

[0032] In step S101, a current depth image of a target livestock and poultry box is acquired by using an image acquisition device pre-installed on a target transport mechanical arm.

[0033] In some embodiments, the current depth image of the target livestock and poultry box 4 is acquired by using the image acquisition device 3 pre-installed on the target transport mechanical arm, including:

[0034] The image acquisition device 3 pre-installed on the target transport mechanical arm is calibrated by a checkerboard to obtain a calibrated image acquisition device 3.

[0035] The current depth image of the target livestock and poultry box 4 is acquired by using the calibrated image acquisition device 3.

[0036] It should be noted that, as shown in Figure 2 , the target transport mechanical arm 1 is provided with a transport box 2, which is arranged at the end of the mechanical arm, and the image acquisition device 3 is arranged on the transport box 2.

[0037] In actual execution, the target transfer robot arm 1 moves the image acquisition device 3 = 3 (such as a structured light depth camera) to a preset height (for example, 20 cm) vertically above the target livestock feed box 4, calibrates the image acquisition device 3 with a chessboard, establishes a depth mapping relationship between the pixel coordinates and the camera coordinate system three-dimensional coordinates, and obtains the calibrated image acquisition device 3. The current depth image of the target livestock feed box 4 (i.e., the image of the remaining feed in the current target livestock feed box 4) is acquired by using the calibrated image acquisition device 3.

[0038] Further, to cope with low light environments, an infrared fill light can be installed above the image acquisition device 3 to provide fill light for the image acquisition process, and a dust cover can also be added to the image acquisition device 3 to prevent feed dust from contaminating the lens.

[0039] In step S102, the region of interest is cropped in the current depth image, and a plurality of data points are extracted in the region of interest to fit the plurality of data points to obtain the bottom reference plane of the target livestock feed box.

[0040] In some embodiments, the region of interest is cropped in the current depth image, and a plurality of data points are extracted in the region of interest to fit the plurality of data points to obtain the bottom reference plane of the target livestock feed box, including:

[0041] The current depth image is edge detected to obtain a binary edge image of the target livestock feed box;

[0042] The binary edge image is peak detected to determine a binary boundary image of the target livestock feed box;

[0043] The region of interest is cropped in the binary boundary image, and a plurality of data points are extracted in the region of interest to fit the plurality of data points to obtain the bottom reference plane of the target livestock feed box.

[0044] In actual execution, the Canny edge detection method is used to perform edge detection on the current depth image to connect weak edges connected with strong edges, so as to obtain a binary edge image of the target livestock and poultry feed box 4. Further, the Hough transform positioning method is used to perform peak value detection on the binary edge image to determine the four vertex coordinates of the target livestock and poultry feed box 4 through straight line intersection points to complete edge positioning and obtain a binary boundary image of the target livestock and poultry feed box 4. Next, the region of interest is intercepted in the binary boundary image, and the RANSAC algorithm is used to randomly select three data points in the region of interest to fit a plane model (normal vector and intercept), and the distance of all other data points to the plane model is calculated. If the distance of the data point to the plane model is less than a preset threshold, it is considered that the data point belongs to the plane model and is included in the inlier set. Through multiple iterations of the above process, the plane model with the largest inlier set is selected as the bottom reference plane of the target livestock and poultry feed box 4.

[0045] In step S103, the residual feed volume in the target livestock and poultry feed box 4 is calculated according to the bottom reference plane and the current depth image.

[0046] In some embodiments, calculating the residual feed volume in the target livestock and poultry feed box according to the bottom reference plane and the current depth image comprises:

[0047] The current depth image is subjected to multi-frame sliding average filtering processing to obtain a smoothed depth image.

[0048] The residual feed volume in the target livestock and poultry feed box is calculated according to the height of each pixel point and the bottom reference plane.

[0049] In actual execution, since the surface of the residual feed in the target livestock and poultry feed box 4 has concave-convex phenomenon, the current depth image is subjected to multi-frame sliding average filtering processing in the embodiment of the application. Specifically, for each frame of depth image, preprocessing (such as denoising, voxel down-sampling, etc.) is performed, and then the depth values at the same position are averaged according to a sliding window (such as 5 frames) to obtain a smoothed depth image. Thus, the measurement error caused by the uneven surface of the feed can be reduced, and the stability and accuracy of the residual feed volume in the target livestock and poultry feed box 4 calculated subsequently can be improved.

[0050] Further, the residual feed volume (i.e. the residual feed volume in the target livestock and poultry feed box 4) is calculated according to the height difference of each pixel point and the bottom reference plane, and the expression is (V = ∑hi × pixel area). In combination with the dynamic calibration mechanism, the transfer mechanical arm 1 is moved to below the discharging equipment of the total feed bin.

[0051] In step S104, the residual feed volume and the preset feeding amount standard are used to generate a feed supplement demand, so that the transfer mechanical arm is loaded with a corresponding amount of feed according to the feed supplement demand.

[0052] like Figure 2 As shown, in actual execution, the feeding demand is generated based on the volume of remaining material and the preset feeding amount standard. By using the linear equation fitted to the test data of the outlet flow rate of the feeding equipment in the total feed bin and the feeding demand, the feeding time corresponding to the required amount of feeding in the total feed bin is calculated. The feeding demand and feeding time are sent to the feeding equipment together. The feeding equipment drives the stepper motor to control the auger speed. At the same time, a closed-loop feedback mechanism is used to feed the material into the transfer box 2 of the transfer robotic arm 1. After each feeding, a weighing device can be set inside the transfer box 2 to weigh the material a second time to verify the actual feeding amount. If the error exceeds ±2%, the coefficients of the linear equation are dynamically corrected until the corresponding amount of feed is loaded into the transfer box 2.

[0053] In step S105, the feed-loaded transfer robotic arm 1 is smoothly moved to the detection pre-pose, and the three-dimensional point cloud of the target livestock cage is captured by the image acquisition device.

[0054] In some embodiments, the feed-loaded transfer robotic arm 1 is smoothly moved to a detection pre-pose, and a three-dimensional point cloud of the target livestock cage is captured by an image acquisition device, including:

[0055] Based on the preset joint space interpolation algorithm and RRT* algorithm, the transfer robot arm loaded with feed is smoothly moved from the loading position to the detection pre-pose.

[0056] The image acquisition device is controlled to emit coded stripes at a preset frequency to capture the three-dimensional point cloud of the target livestock cage.

[0057] In actual execution, based on a preset joint space interpolation algorithm (such as fifth-order polynomial trajectory planning), the transfer robot arm 1 loaded with feed is smoothly moved from the loading position to the detection pre-pose. During this process, the RRT* algorithm is used to detect dynamic obstacles on the path in real time (such as moving rabbit cage supports) to ensure that the end effector carrying the feed box arrives at the observation point at a preset moving speed and at a distance of height from the target cage. Then, the image acquisition device 3 projects coded stripes onto the target livestock cage 5 at a preset frequency to capture the three-dimensional point cloud of the target livestock cage.

[0058] In step S106, a cluster of feed box point clouds is collected in the three-dimensional point cloud of the target livestock cage, and the center coordinates of the minimum bounding box of the feed box are calculated based on the feed box point cloud cluster.

[0059] In some embodiments, a point cloud cluster of feed boxes is acquired in the three-dimensional point cloud of the target livestock cage, and the coordinates of the minimum bounding box center of the feed box are calculated based on the point cloud cluster of the feed box, including:

[0060] Voxel downsampling and statistical outlier filtering are performed on the 3D point cloud of the target livestock cage to obtain the processed 3D point cloud of the target livestock cage.

[0061] segmenting the support plane in the target livestock cage three-dimensional point cloud after the segmentation processing to extract a feed box point cloud cluster;

[0062] performing principal component analysis processing on the feed box point cloud cluster to calculate the minimum bounding box center coordinates of the feed box.

[0063] In actual execution, the target livestock cage three-dimensional point cloud is first voxel down-sampled (such as 2mm voxel grid) and statistical outlier filtering using the PCL library to obtain the target livestock cage three-dimensional point cloud after processing, then the support plane in the target livestock cage three-dimensional point cloud after the segmentation processing is segmented by the RANSAC algorithm to extract a feed box point cloud cluster, and finally the minimum bounding box center coordinates of the feed box are calculated using principal component analysis (PCA).

[0064] In step S107, a spatial straight line trajectory is calculated according to the minimum bounding box center coordinates of the feed box and the detection pre-pose to control the feed loading transfer robot to move to the top of the target livestock feed box and release the feed into the target livestock feed box according to the spatial straight line trajectory.

[0065] In actual execution, the feed box position in the camera coordinate system (i.e. the detection pre-pose) is converted into the robot base coordinate system coordinate through the hand-eye calibration matrix, the Cartesian space straight line trajectory of the transfer robot 1 is planned according to the minimum bounding box center coordinates of the feed box and the robot base coordinate system coordinate, and the feed loading transfer robot 1 is controlled to move to the preset height at the top of the target livestock feed box 4 according to the spatial straight line trajectory. After reaching the corresponding position, the unloading gate of the transfer feed box 2 is triggered to allow the feed in the transfer feed box 2 to be put into the target livestock feed box 4.

[0066] In summary, the large-scale robot livestock feeding method based on machine vision according to the embodiment of the present application has the following beneficial effects:

[0067] (1) The remaining amount in the feed box of each livestock can be accurately identified, so that the feed intake of each livestock can be fully understood, and the amount of feed for each feeding can be accurately calculated and controlled, effectively avoiding the problem of excessive feeding or insufficient feeding caused by inaccurate judgment in traditional manual feeding, significantly reducing feed waste, and ensuring the nutritional balance of livestock;

[0068] (2) The combination of the robot and machine vision recognition not only realizes the automation of the feeding process, greatly reducing the labor intensity, but also quickly responds to the feeding demand of livestock, ensuring that the feeding is carried out at the best time, improving the timeliness and efficiency of feeding;

[0069] (3) Good adaptability and scalability, can easily cope with different types, different growth stages of livestock and poultry feeding needs, through the continuous optimization and upgrading of algorithm, can continuously improve the recognition accuracy and feeding efficiency, meet the long-term development needs of large-scale farms.

[0070] Secondly, the livestock and poultry feeding device based on machine vision of large-scale mechanical arm according to the embodiment of the present application is described with reference to the accompanying drawings.

[0071] Figure 3 It is the block diagram of the livestock and poultry feeding device based on machine vision of large-scale mechanical arm according to the embodiment of the present application.

[0072] As shown in Figure 3 The livestock and poultry feeding device based on machine vision of large-scale mechanical arm 30 includes a collection module 301, a fitting module 302, a volume calculation module 303, a demand generation module 304, a capture module 305, a coordinate calculation module 306 and a feeding module 307.

[0073] The collection module 301 is used to collect the current depth image of the target livestock and poultry box by using the image collection device pre-installed on the target transfer mechanical arm. The fitting module 302 is used to intercept the region of interest in the current depth image, and extract a plurality of data points in the region of interest, so as to fit the plurality of data points to obtain the bottom reference plane of the target livestock and poultry box. The volume calculation module 303 is used to calculate the remaining material volume in the target livestock and poultry box according to the bottom reference plane and the current depth image. The demand generation module 304 is used to generate the replenishment demand according to the remaining material volume and the preset feeding amount standard, so as to load the corresponding amount of feed into the transfer mechanical arm according to the replenishment demand. The capture module 305 is used to smoothly move the transfer mechanical arm loaded with feed to the detection pre-pose, and capture the target livestock cage three-dimensional point cloud by using the image collection device. The coordinate calculation module 306 is used to collect the box point cloud cluster in the target livestock cage three-dimensional point cloud, and calculate the box minimum bounding box center coordinates according to the box point cloud cluster. The feeding module 307 is used to calculate the spatial straight line trajectory according to the box minimum bounding box center coordinates and the detection pre-pose, so as to control the transfer mechanical arm loaded with feed to move to the top of the target livestock and poultry box according to the spatial straight line trajectory, and release the feed into the target livestock and poultry box

[0074] In some embodiments, the collection module 301 includes:

[0075] The calibration unit is used to calibrate the image collection device pre-installed on the target transfer mechanical arm by using the checkerboard, so as to obtain the calibrated image collection device.

[0076] The collection unit is used to collect the current depth image of the target livestock and poultry box by using the calibrated image collection device.

[0077] In some embodiments, the fitting module 302 comprises:

[0078] a first detection unit configured to perform edge detection on the current depth image to obtain a binarized edge image of the target livestock feed box;

[0079] a second detection unit configured to perform peak detection on the binarized edge image to determine a binarized boundary image of the target livestock feed box;

[0080] a fitting unit configured to crop a region of interest in the binarized boundary image and extract a plurality of data points in the region of interest to fit the plurality of data points to obtain a bottom reference plane of the target livestock feed box.

[0081] In some embodiments, the volume calculation module 303 comprises:

[0082] an image processing unit configured to perform multi-frame sliding average filtering on the current depth image to obtain a smoothed depth image;

[0083] a volume calculation unit configured to calculate the residual volume in the target livestock feed box according to the height of each pixel point and the bottom reference plane.

[0084] In some embodiments, the capturing module 305 comprises:

[0085] a motion unit configured to smoothly move the feed loading transfer robot from the loading position to the detection pre-position based on a preset joint space interpolation algorithm and RRT* algorithm;

[0086] a capturing unit configured to control the image acquisition device to emit coded stripes of a preset frequency to capture the target livestock cage three-dimensional point cloud.

[0087] In some embodiments, the coordinate calculation module 306 comprises:

[0088] a point cloud processing unit configured to perform voxel down-sampling and statistical outlier filtering on the target livestock cage three-dimensional point cloud to obtain a processed target livestock cage three-dimensional point cloud;

[0089] a segmentation unit configured to segment the support plane in the processed target livestock cage three-dimensional point cloud to extract a feed box point cloud cluster;

[0090] an analysis processing unit configured to perform principal component analysis on the feed box point cloud cluster to calculate the minimum bounding box center coordinates of the feed box.

[0091] It should be noted that the foregoing explanation and description of the embodiment of the machine vision-based large-scale robotic livestock feeding method also applies to the machine vision-based large-scale robotic livestock feeding device of this embodiment, which will not be described here.

[0092] The scale mechanical arm livestock and poultry feeding device based on machine vision has the following beneficial effects:

[0093] (1) The remaining amount in the feed box of each livestock and poultry can be accurately identified, so that the feed intake of each livestock and poultry is fully understood, and the amount of feed fed each time is accurately calculated and controlled, effectively avoiding the problems of excessive feeding or insufficient feeding caused by inaccurate judgment in traditional manual feeding, significantly reducing feed waste, and ensuring balanced nutrition of livestock and poultry;

[0094] (2) The combination of mechanical arm and machine vision recognition not only realizes the automation of the feeding process, greatly reduces the labor intensity, but also quickly responds to the feeding demand of livestock and poultry, ensures feeding at the best time, and improves the timeliness and efficiency of feeding;

[0095] (3) It has good adaptability and scalability, can easily cope with the feeding needs of different types and different growth stages of livestock and poultry, and through continuous optimization and upgrading of algorithms, can continuously improve the recognition accuracy and feeding efficiency, and meet the long-term development needs of large-scale farms.

[0096] Figure 4 The electronic device provided in the embodiment of the present application is shown in the structural schematic diagram. The electronic device can include:

[0097] The memory 401, the processor 402 and the computer program stored in the memory 401 and executable on the processor 402.

[0098] The processor 402 executes the program to realize the scale mechanical arm livestock and poultry feeding method based on machine vision provided in the above-mentioned embodiments.

[0099] Further, the electronic device further includes:

[0100] The communication interface 403 is used for communication between the memory 401 and the processor 402.

[0101] The memory 401 is used to store the computer program executable on the processor 402.

[0102] The memory 401 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.

[0103] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 4 222In the figure, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0104] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can complete communication between each other through an internal interface.

[0105] The processor 402 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application.

[0106] The embodiment of the present application further provides a computer program product, and the computer program / instruction is executed by the processor to realize the machine vision-based large-scale mechanical arm livestock and poultry feeding method as above.

[0107] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the program is executed by the processor to realize the machine vision-based large-scale mechanical arm livestock and poultry feeding method as above.

[0108] In the description of the application, reference to "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. In addition, the usage of "first", "second" and the like does not indicate any order but rather serves merely to name various components. Moreover, the usage of "top", "bottom", and the like is made for the purpose of illustration only and does not indicate any orientation. The terms "coupled" and "connected", along with their derivatives, can be used. It should be understood that these terms are not intended as synonyms for each other. Rather, particular features are described as being coupled or connected where the feature is in some way present, for example through shared use of one or more components, and can be communicatively, electrically, structurally, and / or mechanically connected, for example. Similarly, "coupled" or "connected" can be used to indicate that two or more members are either directly in contact or indirectly in contact through one or more intermediate members.

[0109] Furthermore, the terms "first", "second", and the like, merely denote different categories, and do not imply a relative importance or a specific order. Thus, features defined with "first", "second" and the like can include at least one of the features, either explicitly or implicitly. In the description of the application, the term "N" means at least two, for example two, three, etc., unless explicitly specified otherwise.

[0110] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments of modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps, and alternate implementations are possible. In some embodiments, the processes or methods described can be accomplished with one or more hardware items, for example, hardwired circuits, memory, logic circuits, look-up tables, microcode or the like, software programs, firmware programs, microcode routines, embedded logic, embedded software, or any combination thereof, which work together to cause a general purpose computer, a special purpose computer, or both, to perform the processes or methods described. The various embodiments further can interact with a user through one or more computer programs, software applications, firmware applications, operating systems, or the like, which interact with a user. Such software can be written in any of a variety of suitable programming languages and can be executed using a variety of suitable hardware and software configurations. It will be appreciated that computer programs, software applications, firmware applications, operating systems, or the like, can be written in any combination of one or more suitable programming languages, and that such software can be executed using one or more computing devices capable of netlist generation as described herein.

[0111] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions stored in a computer readable medium, which can be executed by an instruction execution system, apparatus or device, such as a computer-based system, a processor-based system, or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or a combination of the above. For the purposes of this specification, a "computer readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus or device. The computer readable medium can be a computer readable storage medium or a computer readable signal medium. The computer readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a propagation medium. The computer readable signal medium can include, but is not limited to, a computer readable medium that facilitates transfer of the program from one place to another. A specific example of a computer readable medium is a non-transitory computer-readable storage medium. A specific example of a computer readable signal medium is a source or destination of the computer readable medium. Another specific example of a computer readable signal medium is a computer readable signal travelling through space. Thus, a computer readable medium can take many forms of hardware to carry out the program for use by or in connection with the instruction execution system, apparatus or device.

[0112] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware and in another embodiment, the hardware can be implemented with any or a combination of the following technologies, which are all well known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon an application of data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.

[0113] Those of skill in the art would understand that the steps carried out in the above-mentioned embodiment methods can be carried out by program instructions to relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0114] In addition, each function unit in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0115] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A livestock feeding method using a machine vision-based large-scale robot, characterized by, The method comprises the following steps: acquiring a current depth image of a target feed box of livestock and poultry by using an image acquisition device pre-installed on a target transfer robot arm; cutting out a region of interest in the current depth image and extracting a plurality of data points in the region of interest to fit the plurality of data points to obtain a bottom reference plane of the target feed box of livestock and poultry; calculating a residual feed volume in the target feed box of livestock and poultry according to the bottom reference plane and the current depth image; generating a feed supplement demand according to the residual feed volume and a preset feed amount standard to load a corresponding amount of feed into the transfer robot arm according to the feed supplement demand; smoothly moving the transfer robot arm loaded with feed to a detection pre-pose and capturing a three-dimensional point cloud of a target livestock and poultry cage by using the image acquisition device; acquiring a feed box point cloud cluster in the three-dimensional point cloud of the target livestock and poultry cage and calculating a minimum bounding box center coordinate of the feed box according to the feed box point cloud cluster; calculating a spatial straight line trajectory according to the minimum bounding box center coordinate of the feed box and the detection pre-pose to control the transfer robot arm loaded with feed to move to a position directly above the target feed box of livestock and poultry according to the spatial straight line trajectory and release feed into the target feed box of livestock and poultry.

2. The machine vision based scaled robotic animal feeding method according to claim 1, wherein, The method comprises the following steps: calibrating the image acquisition device pre-installed on the target transfer robot arm by using a chessboard to obtain a calibrated image acquisition device; acquiring the current depth image of the target feed box of livestock and poultry by using the calibrated image acquisition device.

3. The machine vision based scaled robotic arm livestock feeding method according to claim 1, wherein, The method comprises the following steps: performing edge detection on the current depth image to obtain a binary edge image of the target feed box of livestock and poultry; performing peak detection on the binary edge image to determine a binary boundary image of the target feed box of livestock and poultry; cutting out the region of interest in the binary boundary image and extracting a plurality of data points in the region of interest to fit the plurality of data points to obtain the bottom reference plane of the target feed box of livestock and poultry.

4. The machine vision based scaled robotic arm livestock feeding method according to claim 1, wherein, The method comprises the following steps: performing multi-frame sliding average filtering processing on the current depth image to obtain a smoothed depth image; calculating the residual feed volume in the target feed box of livestock and poultry according to the height of each pixel point and the bottom reference plane.

5. The machine vision based scaled robotic arm livestock feeding method according to claim 1, wherein, The method comprises the following steps: based on a preset joint space interpolation algorithm and RRT* algorithm, smoothly moving the transfer robot arm loaded with feed from a loading position to the detection pre-pose; controlling the image acquisition device to emit coded stripes at a preset frequency to capture the three-dimensional point cloud of the target livestock and poultry cage.

6. The machine vision based scaled robotic arm livestock feeding method according to claim 1, wherein, The step of acquiring feed box point cloud clusters in the three-dimensional point cloud of the target livestock cage and calculating the minimum bounding box center coordinates of the feed boxes based on the feed box point cloud clusters includes: Voxel downsampling and statistical outlier filtering are performed on the three-dimensional point cloud of the target livestock cage to obtain the processed three-dimensional point cloud of the target livestock cage. The supporting plane in the processed target livestock cage three-dimensional point cloud is segmented to extract the feed box point cloud cluster; Principal component analysis is performed on the point cloud cluster of the material box to calculate the coordinates of the minimum bounding box center of the material box.

7. A machine vision-based large-scale robotic livestock feeding device, characterized in that, include: The acquisition module is used to acquire current depth images of the target livestock feed box using image acquisition equipment pre-installed on the target transfer robotic arm; The fitting module is used to extract a region of interest in the current depth image and extract multiple data points in the region of interest to fit the multiple data points to obtain the bottom reference plane of the target livestock feed box; The volume calculation module is used to calculate the volume of residual material in the target livestock feed box based on the bottom reference plane and the current depth image; The demand generation module is used to generate a feeding demand based on the volume of the remaining material and the preset feeding amount standard, so as to load the corresponding amount of feed into the transfer robotic arm according to the feeding demand; The capture module is used to smoothly move the feed-loaded transfer robot arm to the detection pre-pose and capture the three-dimensional point cloud of the target livestock cage through the image acquisition device. The coordinate calculation module is used to collect feed box point cloud clusters in the three-dimensional point cloud of the target livestock and poultry cage, and calculate the minimum bounding box center coordinates of the feed box based on the feed box point cloud clusters. The feeding module is used to calculate a spatial straight-line trajectory based on the minimum bounding box center coordinates of the feed box and the detected pre-pose, so as to control the transfer robot arm loaded with feed to move directly above the target livestock feed box according to the spatial straight-line trajectory, and release the feed into the target livestock feed box.

8. An electronic device, comprising: include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the machine vision-based large-scale robotic arm livestock feeding method as described in any one of claims 1-6.

9. A computer program product, characterised in that, When the computer program / instructions are executed by the processor, they implement the machine vision-based large-scale robotic arm livestock feeding method as described in any one of claims 1-6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by a processor to implement the machine vision-based large-scale robotic arm livestock feeding method as described in any one of claims 1-6.

Images