Full-automatic milking robot

Abstract

The invention provides a full-automatic milking robot, which relates to the technical field of robots and specifically comprises a robot master control system, an image recognition system, a mechanical arm control system and a tail end executing mechanism, the robot master control system is responsible for coordinating each part to work according to a certain sequence; the image recognition system is used for acquiring a cattle nipple image, recognizing a cattle nipple, positioning the cattle nipple and transmitting the information to the master control system; the tail end executing mechanism is used for conducting cleaning, medicated bath and rolling brush massage on cow nipples and completing the whole automatic milking process of cup sleeving, milking and cup collecting. The cow nipple health condition rating subsystem divides the health conditions of the cow nipples into four grades of excellent, good, medium and poor, a cow nipple health degree rating model is constructed by using YOLOv5, the cow health conditions and the rationality of a daily milking process can be identified in time by using the model, and the daily life of the cow can be adjusted in time.

Description

Technical Field

[0001] The invention relates to the technical field of robots, and in particular to a fully automatic milking robot. Background Art

[0002] With the improvement of people's living standards and the popularization of health awareness, people's demand for the quantity and quality of fresh milk and milk products is increasing day by day. Therefore, standardization, scale, intensification and modernization have become the trend and goal of the development of my country's dairy farming industry. The fully automatic intelligent milking robot can accurately control the entire process of milking and maximize the milking of cows in a natural state. While ensuring the health of cows, it not only increases the milk production and quality, but also effectively improves work efficiency and saves labor costs. Therefore, it is of great significance to research and develop a fully automatic intelligent milking robot.

[0003] At present, most dairy farms in my country still require workers to perform repetitive operations at their workstations for a long time, which is labor-intensive and has a poor working environment. In addition, improper milking operations can easily lead to milk source pollution. Combining artificial intelligence technology, sensor technology, and robotics technology, they are applied to the dairy farming industry. Developing an automatic intelligent milking robot that can accurately control the entire milking process will promote the standardization, scale, intensification, and modernization of dairy farms. It will free farm workers from repetitive milking operations, save labor costs, reduce the possibility of milk source pollution, and improve the quality of milk, which has broad application prospects. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention provides a fully automatic milking robot;

[0005] A fully automatic milking robot, comprising a robot overall control system, an image recognition system, a mechanical arm control system and an end-effector;

[0006] The robot master control system uses the Jetson Xavier NX core board to recognize images collected by the image recognition system, control the robotic arm to reach a specified position, and control the end actuator. The image recognition system, the robotic arm control system, and the end actuator are all connected to the robot master control system.

[0007] The image recognition system includes a camera and a first servo, wherein the camera specifically uses an Intel RealSense D435i depth camera to identify and locate the nipple of a cow, and the camera is connected to the robot control system via a USB interface; the robot control system outputs a PWM pulse signal to control the first servo for adjusting the angle of the camera;

[0008] The robotic arm control system is connected to a switch with the robot master control system through a LAN network interface, so that the two are in the same local area network; after the robot master control system obtains the coordinates of the cow nipple, it sends the control information to the robotic arm control system through the TCP / IP protocol; the robotic arm control system controls the end of the robotic arm to move to the coordinate origin first, waits for the image recognition system to work, and then controls the milk cup to move to the bottom of the cow nipple to perform the cupping operation, and if it fails, this action will be repeated repeatedly;

[0009] The robotic arm control system also includes an obstacle avoidance sensor, specifically a multi-point laser ranging sensor, which is connected to the robotic arm control system using UART serial port communication and is installed at the end actuator of the robotic arm;

[0010] The end actuator is connected to the GPIO interface of the robot's overall control system, wherein the end actuator includes a drug bath mechanism, a roller brush mechanism, a cylinder control mechanism, and a vacuum milking mechanism; when the end of the robotic arm reaches below the corresponding cow nipple, the end actuator starts to work;

[0011] The medicinal bath mechanism performs medicinal bathing and cleaning treatment on the nipples of the dairy cows;

[0012] The roller brush mechanism is composed of a bracket, a roller, a second steering gear, a motor, connecting parts and a single-chip control board. The motor and the roller brush are connected and installed at the end of the bracket through a coupling. The bracket is located on one side of the milk tray and can rotate around the connecting shaft. The second steering gear and the bracket form a rotating pair, and the bracket is driven to rotate by the second steering gear. The single-chip control board provides an output signal to control the start and stop of the motor and the second steering gear.

[0013] When the robot starts, the output pins of the roller brush and the bracket on one side of the main board are set to low position, located in the initial position, and no operation is performed. After reading the coordinates of the cow's nipples and completing the medicinal bath work, the main board sets the corresponding output pin of the bracket to high position, and the bracket is driven by the second servo to flip to the specified position. After the bracket reaches the specified position, the main board sets the corresponding output pin of the roller brush to high position, and the roller brush starts to rotate through the motor drive, and then moves the roller to the relative specified position of the cow's nipples through the movement of the mechanical arm, and stays for five seconds to complete the cleaning of the cow's nipples. After completing the cleaning of the first cow's nipples, move the mechanical arm back to the initial position, wait for three seconds, and then complete the cleaning of the remaining cow's nipples in turn. After the roller brush mechanism completes the cleaning work, the single-chip microcomputer control board sets the corresponding control pin of the roller brush to low position, and the roller brush stops working. After the roller brush stops working, the single-chip microcomputer control board sets the corresponding pin of the bracket to low position, and the second servo drives the bracket to flip back to the initial position, and the single-chip microcomputer control board outputs the "rolling brush completed" message prompt.

[0014] The cylinder control mechanism includes a first air pump, a solenoid valve, a relay, a single-chip control board, a cylinder and a power supply; the cylinder is installed in a vacuum milking mechanism at the end of a robotic arm, the single-chip control board controls the opening and closing of the relay, the relay controls the on and off of the solenoid valve, the first air pump is connected to an air storage tank through an air pipe, the air storage tank is then connected to the air inlet of the solenoid valve through an air pipe, and the air outlet of the solenoid valve is connected to the cylinder through an air pipe.

[0015] The vacuum milking mechanism is provided with four milk cups corresponding to the four nipples of the cow respectively. The upper part of the milk cup is made of soft rubber material. The vacuum milking mechanism also includes a second air pump. The second air pump is used to extract air to create a vacuum environment so that the milk cup is adsorbed on the nipple, and the cylinder is controlled to move at a set frequency for automatic milking. After the milking is completed, the second air pump is controlled to release the milk cup from the cow's nipple.

[0016] The image recognition system also includes a data integration system, which stores daily photos of the cow's udders, the cow's daily milking volume, and the rating of the cow's nipple health status; specifically, the YOLOv5 algorithm is used to identify the cow's nipples. After the coordinates of the cow's nipples in the image are identified, the real coordinates of the cow's nipples in the three-dimensional world are calculated by combining the depth information of the depth camera and the rotation and translation matrix;

[0017] The rating of the health status of the cow's teats is specifically as follows: the health status of the cow's teats is divided into four levels: level one is high-quality teats, level two is benign teats, level three is neutral teats, and level four is diseased teats; the high-quality teats have no nipple protrusions, no color difference, no surface lesions, no abnormalities inside the teats, and no lacerations or ulcers on the tip of the teats; the end of the benign teat is surrounded by a ring of protrusions no more than 1 mm, without protrusions formed by old keratin, and the surface is smooth; the end of the neutral teat is surrounded by 1 to 3 mm old keratin protrusions or raised nodules, and the surface is rough; the diseased teats and neutral teats are The end is surrounded by old horny protrusions or raised nodules larger than 4 mm, and the nipples are flower-shaped or cracked. A training data set is collected in advance. The training data set is photos of cow nipples taken on the ranch, which are graded and labeled manually according to the rating standards. Rating labels are labeled based on the training data set, and the YOLOv5 algorithm is used to train a cow nipple health rating model. The model obtains the rating results of the corresponding nipples based on the input cow nipple pictures or videos. The trained model is used to detect the collected daily udder photos of cows and rate the health status of cow nipples.

[0018] The control system of the milking robot also includes a graphical interface including a camera preview module, a functional module, and a display module. The camera preview module is used to display the video image returned by the camera in real time; the functional module is used to control the image recognition system and the start and stop of the mechanical arm control system and the end actuator, and display the data integration system to perform integrated analysis on the health status and milking volume of the cow; the display module is used to display the current movement position of the mechanical arm in real time, and to manually control the mechanical arm;

[0019] The fully automatic milking robot is used to perform the following steps:

[0020] Step 1: The image recognition system detects the position of the cow's teat and calculates the coordinates of the target point of the robot arm's movement;

[0021] Step 2: The fully automatic milking robot controls the mechanical arm to bathe the cow's nipples, cleans them with a roller brush, controls the cylinder to complete the cup-standing and cup-sleeving movements, and performs milking;

[0022] Step 3: During the milking process, a flow meter is used to detect the milking volume. When the change in the milking volume within the set time is less than the set threshold, the cylinder releases the milk cup to end the milking, and the robotic arm returns to the origin to complete the milking.

[0023] The cupping is performed one by one: first, the cupping of nipple No. 1 is performed, and when the cupping of nipple No. 1 is successful, the robot arm returns to the target detection origin for detection, calculates the coordinates of nipple No. 2, and then the cupping of nipple No. 2, and then the cupping of nipples No. 3 and No. 4 in sequence. The milk cup is placed horizontally when the cupping is not performed. When the cupping action is performed, a rope is connected to the lower end of the milk cup, and the other end of the rope is connected to the motor. By driving the motor to pull the rope closer, the milk cup will be pulled into the milk cup groove designed on the robot arm under the action of the pulling force, and then stand up.

[0024] During the cupping process, the image recognition system determines whether the cupping is successful by the number of nipples in the image, and the cupping process stops and returns to the starting point for recognition.

[0025] During the milking process, if a cup falls off, the vacuum pump corresponding to the milk cup will stop working, and the milk cup will return to the initial upright state. At the same time, it is determined that the milking of the nipple has failed, and other milk cups will continue to work normally until all flow meter readings no longer change;

[0026] The beneficial effects of adopting the above technical solution are:

[0027] The present invention provides a fully automatic milking robot which can accurately control the entire milking process and realize milking of cows in a natural state to the greatest extent. While ensuring the health of cows, it not only increases the milk yield and quality, but also effectively improves work efficiency, saves labor costs, and is conducive to promoting the refined development of the dairy industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a general structural block diagram of the milking robot system of the present invention;

[0029] Figure 2 It is the overall working flow chart of the system of the present invention;

[0030] Figure 3 It is a workflow diagram of the image recognition system of the present invention;

[0031] Figure 4 This is a working diagram of the robot control system of the present invention;

[0032] Figure 5 It is the obstacle avoidance working flow chart of the present invention;

[0033] Figure 6 This is a flow chart of the working process of the end actuator of the present invention;

[0034] Figure 7 A flow chart of the milking process of the system of the present invention;

[0035] Figure 8 This is a schematic diagram of the graphical interface of the present invention;

[0036] Fig. 9 This is a schematic diagram of the bovine teat health grade detection method of the present invention; DETAILED DESCRIPTION

[0037] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0038] A fully automatic milking robot, such as Figure 1 As shown, it includes the robot overall control system, image recognition system, mechanical arm control system and end effector;

[0039] The robot's overall control system is mainly responsible for coordinating the various parts to work in a certain order; the image recognition system is used to collect images of cow nipples, identify cow nipples, locate cow nipples, and convert the coordinates of the cow nipples in the camera coordinate system into coordinates in the three-dimensional world, and pass them to the overall control system for subsequent cupping and other actions. At the same time, the image recognition system also includes a data integration system; after receiving the coordinates of the cow nipples, the robotic arm control system moves the end of the robotic arm to the bottom of the cow nipples. During the movement of the robotic arm, it can dynamically avoid obstacles and select a suitable path; the actuator at the end of the robotic arm is used to clean, bathe, and massage the cow nipples with a roller brush, and complete the entire automated milking process of cupping, milking, and collecting cups.

[0040] The robot master control system uses the Jetson Xavier NX core board, whose main tasks include collecting images and identifying them, controlling the robot arm to reach the specified position, and controlling the end actuator. Each subsystem is connected to the main control board in different ways, so that the robot master control system can transmit control information to each subsystem in a timely manner.

[0041] The image recognition system includes a camera and a first steering engine. The working process of the image recognition system is as follows: Figure 3 As shown, the camera specifically uses an Intel RealSense D435i depth camera to identify and locate the cow's nipples, and the camera is connected to the robot's overall control system via a USB interface; the robot's overall control system outputs a PWM pulse signal to control the first servo, which is used to adjust the angle of the camera so that the camera can fully see the four nipples of the cow when facing cows of different heights;

[0042] The robot control system is connected to a switch through a LAN network interface and the robot master control system at the same time, so that the two are in the same local area network; the working process of the robot control system is as follows Figure 4 After obtaining the coordinates of the cow's nipple, the robot's overall control system sends the control information to the robotic arm control system through the TCP / IP protocol; the robotic arm control system controls the end of the robotic arm to move to the coordinate origin first, waits for the image recognition system to work, and then controls the milk cup to move to the bottom of the cow's nipple to perform the cupping operation. If it fails, this action will be repeated repeatedly;

[0043] The robotic arm control system also includes an obstacle avoidance sensor, specifically a multi-point laser ranging sensor, which is connected to the robotic arm control system using UART serial port communication and is installed at the end actuator of the robotic arm;

[0044] The robot control system may collide with obstacles such as bull legs during movement, so it has an obstacle avoidance function. The overall process of the obstacle avoidance function is as follows: Figure 5As shown. The sensor used for obstacle avoidance is a multi-point laser ranging sensor, which is connected to the robot control system using UART serial communication and installed in front of the end actuator of the robot. When the robot moves under the nipple, the obstacle avoidance sensor will detect the distance to the obstacle in real time. If there is no obstacle, it will move under the nipple to cup. If there is an obstacle, it will return to the identification point and identify it again.

[0045] The end actuator is connected to the GPIO interface of the robot's overall control system and outputs high and low levels for control. The end actuator includes a drug bath mechanism, a roller brush mechanism, a cylinder control mechanism, and a vacuum milking mechanism. The overall workflow of this system is as follows: Figure 2 When the end of the robot arm reaches below the corresponding nipple, the end actuator starts to work, as shown in Figure 6 shown.

[0046] Before the milking process, in order to meet various requirements such as the robot being able to normally realize the cupping work, collect high-quality milk and prevent the cow's nipple lesions, the medicinal bath mechanism is required to perform medicinal bathing and cleaning treatment on the cow's nipples;

[0047] The roller brush mechanism is composed of a bracket, a roller, a second steering gear, a motor, connecting parts and a single-chip control board. The motor and the roller brush are connected and installed at the end of the bracket through a coupling. The bracket is located on one side of the milk tray and can rotate around the connecting shaft. The second steering gear and the bracket form a rotating pair, and the bracket is driven to rotate by the second steering gear. The single-chip control board provides an output signal to control the start and stop of the motor and the second steering gear.

[0048] When the robot starts, the output pins of the roller brush and the bracket on one side of the main board are set to low position, located in the initial position, and no operation is performed. After reading the coordinates of the cow's nipples and completing the medicinal bath work, the main board sets the corresponding output pin of the bracket to high position, and the bracket is driven by the second servo to flip to the specified position. After the bracket reaches the specified position, the main board sets the corresponding output pin of the roller brush to high position, and the roller brush starts to rotate through the motor drive, and then moves the roller to the relative specified position of the cow's nipples through the movement of the mechanical arm, and stays for five seconds to complete the cleaning of the cow's nipples. After completing the cleaning of the first cow's nipples, move the mechanical arm back to the initial position, wait for three seconds, and then complete the cleaning of the remaining cow's nipples in turn. After the roller brush mechanism completes the cleaning work, the single-chip microcomputer control board sets the corresponding control pin of the roller brush to low position, and the roller brush stops working. After the roller brush stops working, the single-chip microcomputer control board sets the corresponding pin of the bracket to low position, and the second servo drives the bracket to flip back to the initial position, and the single-chip microcomputer control board outputs the "rolling brush completed" message prompt.

[0049] The cylinder control mechanism includes a first air pump, a solenoid valve, a relay, a single-chip control board, a cylinder and a power supply; the cylinder is installed in the vacuum milking mechanism at the end of the mechanical arm, the single-chip control board controls the opening and closing of the relay, the relay controls the on and off of the solenoid valve, the first air pump is connected to the air storage tank through an air pipe, the air storage tank is connected to the air inlet of the solenoid valve through an air pipe, and the air outlet of the solenoid valve is connected to the cylinder through an air pipe. It is responsible for providing the compressed air required by the system, and it compresses the outside air and stores it in the air storage tank and transports it to the system. The cylinder is an actuator that generates movement by receiving compressed air. The solenoid valve is a control element used to control the flow of compressed air. Through the control of electrical signals, the solenoid valve can open or close the air path, thereby controlling the action of the cylinder. The relay, as a lower computer, outputs appropriate high and low voltages to the pins of the solenoid valve to control the opening and closing of the corresponding solenoid valve. The main control board, as an upper computer, connects to the relay of the lower computer, controls the output of the relay level, and then controls the opening and closing of the solenoid valve. The power supply is responsible for supplying power to the relay.

[0050] The vacuum milking mechanism is provided with 4 milk cups, which correspond to the 4 nipples of the cow respectively, so that milking can be achieved simultaneously and work efficiency can be improved. The upper part of the milk cup is made of soft rubber material, which is convenient for being put on the nipples of the cows for milking, and can also reduce the harm to the cows. The vacuum milking mechanism also includes a second air pump, which uses the second air pump to extract air to create a vacuum environment so that the milk cup is adsorbed on the nipples, and at the same time controls the cylinder to move at a set frequency so that the milk cup can achieve a pulsating effect of loosening and tightening to imitate the process of manual milking for automatic milking; after the milking is completed, the second air pump is controlled to loosen the milk cup from the cow's nipples.

[0051] The image recognition system also includes a data integration system to facilitate the management of dairy cows on the ranch. The data integration system stores daily photos of the cows' udders for observing the health of the cows; the daily milking volume of the cows for the management of the ranch's output; and the rating of the health of the cows' nipples. Specifically, the YOLOv5 algorithm is used to identify the cow's nipples. After the coordinates of the cow's nipples in the image are identified, the real coordinates of the cow's nipples in the three-dimensional world are calculated by combining the depth information of the depth camera and the rotation and translation matrix.

[0052] The rating of the health status of the cow's teats is specifically as follows: the health status of the cow's teats is divided into four levels: level one is high-quality teats, level two is benign teats, level three is neutral teats, and level four is diseased teats; the high-quality teats have no nipple protrusions, no color difference, no surface lesions, no abnormalities inside the teats, and no lacerations or ulcers on the tip of the teats; the end of the benign teat is surrounded by a ring of protrusions no larger than 1 mm, there are no protrusions formed by old keratin, and the surface is smooth; the end of the neutral teat is surrounded by 1 to 3 mm old keratin protrusions or raised nodules, and the surface is rough; the end of the neutral teat of the diseased teat is surrounded by old keratin protrusions or raised nodules of more than 4 mm, and the nipple appears flower-shaped or cracked; a training data set is collected in advance, and the training data set is a photo of the cow's teats taken on the ranch, which is manually graded and labeled according to the rating standards, and mixed labeling by multiple people reduces the influence of subjectivity. Rating labels are annotated according to the training data set, and the cow teat health rating model is trained using the YOLOv5 algorithm. The model obtains the rating results of the corresponding teats based on the input cow teat pictures or videos; the trained model is used to detect the collected daily udder photos of cows and rate the health status of cow teats, such as Fig. 9 shown.

[0053] The control system of the milking robot also includes a graphical interface including a camera preview module, a functional module, and a display module. The camera preview module is used to display the video image returned by the camera in real time; the functional module is used to control the image recognition system and the start and stop of the mechanical arm control system and the end actuator, and display the data integration system to perform integrated analysis on the health status and milking volume of the cow; the display module is used to display the current movement position of the mechanical arm in real time, and to manually control the mechanical arm;

[0054] In this embodiment, the graphical interface is as follows Figure 8 As shown, it is divided into three parts: left, middle and right: the left camera preview box, in this column you can see the real-time video returned by the camera. The middle is the main function area, which is responsible for controlling the start of the image detection system and the start of milking. At the same time, you can also open the data integration system, which is responsible for integrating and analyzing the health status and milking volume of the cow. The lower middle part is the current position display area of ​​the robotic arm, which is responsible for real-time display of the current movement position of the robotic arm. The right side is the robotic arm control area, which is used for debugging or special situations, such as hitting an obstacle or abnormal operation, to manually control the robotic arm.

[0055] The fully automatic milking robot is used to perform the following steps: Figure 7 As shown, specifically including:

[0056] Step 1: The image recognition system detects the position of the cow's teat and calculates the coordinates of the target point of the robot arm's movement;

[0057] Step 2: The fully automatic milking robot controls the mechanical arm to bathe the cow's nipples, cleans them with a roller brush, controls the cylinder to complete the cup-standing and cup-sleeving movements, and performs milking;

[0058] Step 3: During the milking process, a flow meter is used to detect the milking volume. When the change in the milking volume within the set time is less than the set threshold, the cylinder releases the milk cup to end the milking, and the robotic arm returns to the origin to complete the milking.

[0059] In this embodiment, the flow meter is responsible for detecting the current milking amount of each nipple. When the flow meter detects that the flow rate (milking amount) no longer changes significantly within a period of time, a signal is sent to the main control board. After receiving the signal, the main control board issues a control instruction through the GPIO interface to separate the milk cup from the cow's nipple and complete the milking process. RS485 serial port communication is used between the flow meter and the Jetson Xavier NX core board. Before use, the corresponding baud rate, data bit, stop bit, check bit and other parameters need to be set. After the setting is completed, the data collected by the flow meter is read at a small time interval and the corresponding solution is performed to obtain the current flow data. By comparing with the flow data collected at the previous time interval, when the difference between the two is less than a certain value (a very small value set artificially), it means that the milking amount has not changed significantly during this period of time, and it can be considered that the current milking work can be completed.

[0060] The whole process of the robot arm movement: the robot arm returns to the coordinate origin → the camera recognizes the teat → the robot arm moves to the cup → the host sends a GPIO signal to control the solenoid valve vacuum pump for vacuum milking → the flow meter detects whether the milking is finished → the robot arm returns to the coordinate origin.

[0061] The principle of cupping is: first, the mouth of the milk cup should be located directly below the nipple of the cow, and the distance between the two should be relatively close. The position is given by the target detection program, and the robot arm movement program controls the robot arm movement to reach it. After that, the GPIO port on the Jetson Xavier NX core board is used to control the opening and closing of the relay to control the opening and closing of the solenoid valve, thereby achieving control of the compressed air flow direction of the air pump. The air pump extracts air to create a vacuum environment so that the milk cup is adsorbed on the nipple to complete the cupping. The solenoid valve that controls the milk cups 1 to 4 to complete the cupping is connected to the relay, and then controlled by pins 17, 27, 22, and 18 on the core board respectively. The milking process imitates manual milking by controlling the rhythmic movement of the cylinder to make the milk cup achieve a pulsating effect of loosening and tightening. This process is controlled by pins 23, 19, 26, and 20 respectively. When the milk cup is put on the nipple, the Jetson Xavier NX main control board will output a GPIO signal to control the opening and closing of the relay, and the relay will then control the start of the vacuum pump. After the vacuum pump is started, negative air pressure will be formed in the milk cup, causing the milk cup to be attached to the nipple and provide pulsed suction to perform the milking action.

[0062] In order to achieve accurate positioning and cupping of all four milk cups, the cupping is performed one by one: first cupping of teat No. 1 is performed, and when teat No. 1 is successfully cupped, the robot arm returns to the target detection origin for detection, calculates the coordinates of teat No. 2, and then cupping of teat No. 2, and then cupping of teat No. 3 and No. 4 in sequence. When the milk cup is not cupped, it is placed horizontally to reduce the space occupied by the upper part of the robot arm, prevent collision with cows or other objects during movement, and improve safety. When performing the cupping action, a rope is connected to the lower end of the milk cup, and the other end of the rope is connected to the motor. By driving the motor to pull the rope closer, the milk cup will be pulled into the designed milk cup groove on the robot arm under the action of tension, and then stand up.

[0063] During the cupping process, the image recognition system determines whether the cupping is successful based on the number of cow nipples in the image. For example, after the first cow nipple is cupped, if four cow nipples can still be recognized, it means that the cupping has failed. If the cupping fails, the cupping process stops and returns to the starting point for recognition.

[0064] During the milking process, if the cup falls off due to insufficient suction, the vacuum pump corresponding to the milk cup will stop working, and the milk cup will return to the initial upright state. At the same time, it is determined that the milking of the nipple has failed, and other milk cups will continue to work normally until all flow meter readings no longer change;

[0065] The working process of the milking robot is as follows Figure 7 As shown in the figure. The image acquisition system is located at the forefront, using the IntelRealSense D435i depth camera to obtain video information in real time. At the same time, the depth camera can also return depth information. A suitable frame is intercepted from the acquired video stream as an image to record the health status of the cow's udder on that day and stored in the data system, and its health rating is also performed. The video stream and depth information are input into the target detection system. The core of the target detection system is the YOLOv5 program, which uses a pre-trained model to perform real-time detection of the video and identify the position of the cow's nipple and the detection frame. The position of the detection frame here refers to the two-dimensional (x, y) plane coordinates of the cow's nipple on this frame of the image, while the three-dimensional (x, y, z) world coordinates are required for the movement of the robotic arm. The coordinate solution system performs corresponding coordinate conversion based on the two-dimensional coordinates obtained by the target detection program and the depth information returned by the depth camera, and finally obtains the three-dimensional world coordinates. Then, the three-dimensional coordinates are input into the robotic arm control system to control the robotic arm to move to the target point and start milking.

[0066] Among them, the work performed by the drug bath mechanism and the milk brush mechanism belongs to the preparation work before formal milking, and its purpose is to ensure that the udder and nipple of the cow are clean and hygienic, so as to prevent bacterial infection and contamination of dairy products. The drug bath mechanism and the milk brush mechanism are driven by an aluminum alloy bracket. When working, the bracket rotates and drives it to the specified position. After the work is completed, the bracket rotates in the opposite direction again to drive it back to the initial position, reducing the occupation of the milking workspace. The stepper motor driver that drives the drug bath mechanism is controlled by pin 10 on the core board, the stepper motor driver of the milk brush mechanism is controlled by pin 24, and the stepper motor driver of the bracket is controlled by pin 7. When the four milk cups are not working (no cupping), they are in a tilted state, that is, placed horizontally. Only when the milk cups need to be cupped will they stand upright under the joint action of the pull rope mechanism and the cup standing mechanism. The stepper motor drivers of the rope pulling mechanism of the milk cups 1 to 4 are controlled by pins 12, 16, 6, and 13 on the core board, respectively, and the stepper motor drivers of the cup standing mechanism of the milk cups 1 to 4 are controlled by pins 25, 9, 11, and 8 on the core board, respectively. When the milk cup is stood up by the rope pulling mechanism and the cup standing mechanism, the cup setting work can begin.

[0067] The solenoid valve used in the milking mechanism is not directly driven by the low output voltage of the GPIO port on the Jetson Xavier NX core board. Therefore, a relay module is added in the middle. The GPIO port on the core board is first connected to the relay and then to the solenoid valve. The solenoid valve is powered by an independent power supply. The high and low levels output by the GPIO port control the switch of the relay, control the opening and closing of the solenoid valve, and indirectly control the movement of the cylinder. The relay is directly connected to the GPIO pin through a DuPont line.

[0068] In addition, circuit isolation through relays can prevent circuit short circuits. The high-voltage circuit is the cylinder power supply circuit, the live wire is connected in series with the normally-off ends of the relay, and the low-voltage circuit consists of the control motherboard output pin and the relay control part. In extreme cases, a circuit short circuit failure will only damage the relay, not the cylinder and the control motherboard, reducing the damage caused by accidents. Subsequent maintenance is also handled around the relay, reducing design costs while improving the reliability of the entire robot control system.

[0069] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.

Claims

1. A fully automatic milking robot, characterized in that: It includes the robot overall control system, image recognition system, robotic arm control system and end effector; The robot master control system uses the Jetson Xavier NX core board to recognize images collected by the image recognition system, control the robotic arm to reach a specified position, and control the end actuator. The image recognition system, the robotic arm control system, and the end actuator are all connected to the robot master control system; The image recognition system includes a camera and a first servo, wherein the camera specifically uses an Intel RealSense D435i depth camera to identify and locate the nipple of a cow, and the camera is connected to the robot control system via a USB interface; the robot control system outputs a PWM pulse signal to control the first servo for adjusting the angle of the camera; The robotic arm control system is connected to a switch with the robot master control system through a LAN network interface, so that the two are in the same local area network; after the robot master control system obtains the coordinates of the cow nipple, it sends the control information to the robotic arm control system through the TCP / IP protocol; the robotic arm control system controls the end of the robotic arm to move to the coordinate origin first, waits for the image recognition system to work, and then controls the milk cup to move to the bottom of the cow nipple to perform the cupping operation, and if it fails, this action will be repeated repeatedly; The end-actuator is connected to the GPIO interface of the robot's overall control system. When the end of the robotic arm reaches below the corresponding cow nipple, the end-actuator starts to work.

2. A fully automatic milking robot according to claim 1, characterized in that: The robotic arm control system also includes an obstacle avoidance sensor, specifically a multi-point laser ranging sensor, which is connected to the robotic arm control system using UART serial port communication and is installed on the end actuator of the robotic arm.

3. The fully automatic milking robot according to claim 1, characterized in that: The end actuator includes a drug bath mechanism, a roller brush mechanism, a cylinder control mechanism, and a vacuum milking mechanism; the drug bath mechanism performs drug bathing and cleaning treatment on the teats of dairy cows; The roller brush mechanism is composed of a bracket, a roller, a second steering gear, a motor, connecting parts and a single-chip control board. The motor and the roller brush are connected and installed at the end of the bracket through a coupling. The bracket is located on one side of the milk tray and can rotate around the connecting shaft. The second steering gear and the bracket form a rotating pair, and the bracket is driven to rotate by the second steering gear. The single-chip control board provides an output signal to control the start and stop of the motor and the second steering gear. When the robot is started, the output pins of the roller brush and the bracket on one side of the main board are set to a low position, located in the initial position, and no operation is performed; after reading the coordinates of the cow's nipples and completing the medicated bath work, the main board sets the corresponding output pin of the bracket to a high position, and the bracket is driven by the second servo to flip to the specified position; after the bracket reaches the specified position, the main board sets the corresponding output pin of the roller brush to a high position, and the roller brush starts to rotate through the motor drive, and then the roller is moved to the relative specified position of the cow's nipples through the movement of the mechanical arm, and stays for five seconds to complete the cleaning of the cow's nipples; after completing the cleaning of the first cow's nipples, the mechanical arm is moved back to the initial position, and after waiting for three seconds, the cleaning of the remaining cow's nipples is completed in turn; after the roller brush mechanism completes the cleaning work, the single-chip microcomputer control board sets the corresponding control pin of the roller brush to a low position, and the roller brush stops working. After the roller brush stops working, the single-chip microcomputer control board sets the corresponding pin of the bracket to a low position, and the second servo drives the bracket to flip back to the initial position, and the single-chip microcomputer control board outputs a "rolling brush completed" message prompt; The cylinder control mechanism includes a first air pump, a solenoid valve, a relay, a single-chip control board, a cylinder and a power supply; the cylinder is installed in the vacuum milking mechanism at the end of the mechanical arm, the single-chip control board controls the opening and closing of the relay, the relay controls the on and off of the solenoid valve, the first air pump is connected to the air storage tank through an air pipe, the air storage tank is connected to the air inlet of the solenoid valve through an air pipe, and the air outlet of the solenoid valve is connected to the cylinder through the air pipe; The vacuum milking mechanism is provided with four milk cups corresponding to the four nipples of the cow respectively. The upper part of the milk cup is made of soft rubber material. The vacuum milking mechanism also includes a second air pump. The second air pump is used to extract air to create a vacuum environment so that the milk cup is adsorbed on the nipple, and the cylinder is controlled to move at a set frequency for automatic milking. After the milking is completed, the second air pump is controlled to release the milk cup from the cow's nipple.

4. The fully automatic milking robot according to claim 1, characterized in that: The rating of the health status of the cow's teats is specifically as follows: the health status of the cow's teats is divided into four levels: level one is high-quality teats, level two is benign teats, level three is neutral teats, and level four is diseased teats; the high-quality teats have no teat protrusions, no color difference, no surface lesions, no abnormalities inside the teats, and no lacerations or ulcers on the teat tips; the benign teats are surrounded by a ring of protrusions no larger than 1 mm, have no protrusions formed by old keratin, and have a smooth surface; the neutral teats are surrounded by 1 to 3 mm old keratin protrusions or The raised nodules have a rough surface; the end of the neutral nipple of the diseased nipple is surrounded by old keratin protrusions or raised nodules larger than 4 mm, and the nipple is flower-shaped or cracked; a training data set is collected in advance, and the training data set is photos of cow nipples taken on the ranch, which are graded and labeled manually according to the rating standards. Rating labels are labeled based on the training data set, and the cow nipple health rating model is trained using the YOLOv5 algorithm. The model obtains the rating results of the corresponding nipples based on the input cow nipple pictures or videos; The trained model is used to detect the daily udder photos of dairy cows and rate the health of the cows’ teats.

5. The fully automatic milking robot according to claim 1, characterized in that: The image recognition system also includes a data integration system, which stores daily udder photos of the cows, the daily milking amount of the cows, and the rating of the health status of the cows' teats; Specifically, the YOLOv5 algorithm is used to identify the cow's nipple. After the coordinates of the cow's nipple in the image are identified, the depth information of the depth camera and the rotation and translation matrix are combined to calculate the real coordinates of the cow's nipple in the three-dimensional world.

6. The fully automatic milking robot according to claim 1, characterized in that: The fully automatic milking robot also includes a graphical interface including a camera preview module, a functional module, and a display module. The camera preview module is used to display the video image returned by the camera in real time; the functional module is used to control the image recognition system and the start and stop of the robotic arm control system and the end actuator, and display the data integration system to perform integrated analysis on the health status and milking volume of the cow; the display module is used to display the current movement position of the robotic arm in real time, and to manually control the robotic arm.

7. The fully automatic milking robot according to claim 1, characterized in that: The fully automatic milking robot is used to perform the following steps: Step 1: The image recognition system detects the position of the cow's teat and calculates the coordinates of the target point of the robot arm's movement; Step 2: The fully automatic milking robot controls the mechanical arm to bathe the cow's nipples, cleans them with a roller brush, controls the cylinder to complete the cup-standing and cup-sleeving movements, and performs milking; Step 3: During the milking process, a flow meter is used to detect the milking volume. When the change in the milking volume within the set time is less than the set threshold, the cylinder releases the milk cup to end the milking, and the robotic arm returns to the origin to complete the milking.

8. The fully automatic milking robot according to claim 7, characterized in that: In step 3, the cupping is performed one by one: first, the cupping of nipple No. 1 is performed, and when the cupping of nipple No. 1 is successful, the mechanical arm returns to the target detection origin for detection, calculates the coordinates of nipple No. 2, and then the cupping of nipple No. 2, and then the cupping of nipples No. 3 and No. 4 are performed in sequence; the milk cup is placed horizontally when the cupping is not performed, and when the cupping action is performed, a rope is used to connect the lower end of the milk cup, and the other end of the rope is connected to the motor, and the rope is pulled closer by the driving motor, and the milk cup will be pulled into the milk cup groove designed on the mechanical arm under the action of tension, and then stand up; During the cupping process, the image recognition system determines whether the cupping is successful by the number of nipples in the image, and the cupping process stops and returns to the starting point for recognition; During the milking process, if a cup falls off, the vacuum pump corresponding to the milk cup will stop working, and the milk cup will return to the initial upright state. At the same time, it is determined that the nipple has failed in milking, and other milk cups will continue to work normally until all flow meter readings no longer change.

Images