Self-adaptive intelligent control system of milking robot

Abstract

The invention provides a self-adaptive intelligent control system of a milking robot, and relates to the technical field of robots. Comprising an upper computer control unit, a dairy cow identification module, a visual identification module, a mechanical arm motion control module, a sensing monitoring module and a lower relay execution module. The system performs distributed cooperative control based on a robot operating system, each module operates in an independent function node form, and data interaction and state synchronization are realized through an ROS message mechanism. The self-adaptive intelligent adjustment strategy is realized through real-time updating of the point position of the moving target of the manipulator and real-time obstacle avoidance adjustment. A double-route fusion algorithm of visual identification and coordinate calibration is adopted, and real-time tracking and high-precision positioning of a dynamic target (dairy cow nipple) are achieved. According to the control system disclosed by the invention, a closed-loop control logic of'identification-decision-execution-feedback 'is adopted, and an integrated closed-loop process of'identification-cleaning-stimulation-adsorption-milking-disinfection' is constructed.

Description

Technical Field

[0001] This invention relates to the field of robotics technology, and in particular to an adaptive intelligent control system for a milking robot. Background Technology

[0002] With the continued growth of global dairy consumption, the dairy farming industry is facing three major pain points: labor shortage, high costs, and quality fluctuations. Traditional herringbone or rotary milking parlors rely on manual cup application and inspection, with skilled workers working two shifts in the early morning, resulting in high labor intensity and a high risk of mastitis due to operational differences, leading to excessive somatic cell counts in raw milk. The EU's 2025 regulations have reduced the somatic cell count limit from 400,000 to 300,000 per mL, forcing farms to improve hygiene standards through precise and unmanned methods. At the same time, the scale of dairy cows is accelerating, with farms having 10,000 cows accounting for more than 35%, requiring efficient milking of thousands of cows per shift, which manual methods can no longer meet the demands of 24-hour continuous production.

[0003] Although milking robots from European and American manufacturers have been deployed in Chinese dairy farms, there are still issues with their effectiveness in adapting to local conditions.

[0004] (1) Static models are difficult to adapt to the working conditions of Chinese Holstein cattle with large differences in body size and unstable gait, resulting in a leakage and cup drop rate of >8%;

[0005] (2) Single visual positioning fails to recognize under the interference of manure and water vapor, causing the robotic arm to collide with the teat, triggering stress in dairy cows and reducing milk production by 10%-15%;

[0006] (3) Existing PID control cannot compensate for disturbances caused by vacuum pulsation and sudden changes in milk flow online. The vacuum fluctuation at the nipple end is ±6kPa, which is higher than the national standard of ±3kPa, thus increasing the risk of mastitis.

[0007] (4) The system is closed and cannot be connected to domestic TMR, estrus monitoring and DHI big data platforms, forming an information island and making it difficult to achieve intelligent management and control throughout the entire life cycle.

[0008] Therefore, developing an adaptive intelligent control system for milking robots with self-learning and self-optimization capabilities has become a bottleneck to be overcome, a key to achieving domestic substitution of high-end agricultural machinery, and ensuring the revitalization of the dairy industry. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides an adaptive intelligent control system for a milking robot. It aims to construct a multi-layered control system integrating perceptual intelligence, behavioral intelligence, and decision-making intelligence to achieve high-precision milking operations and full lifecycle data management in complex dynamic environments. This invention is an agricultural service-oriented mechatronics device—an adaptive intelligent milking robot—that enables automated, precise, and efficient milking operations for dairy cows. This device integrates multiple disciplines such as mechanics, electrical engineering, computer vision, and information processing, enabling it to complete the entire process from cow identification, positioning, teat detection, and autonomous milking by the robotic arm without human intervention.

[0010] The technical solution of the present invention is as follows:

[0011] On the one hand, the present invention provides an adaptive intelligent control system for a milking robot, including a host computer control unit, a cow identification module, a vision recognition module, a robotic arm motion control module, a sensor monitoring module, and a lower-level relay execution module; each module operates as an independent node, including a cow number reading node, a cow teat identification node, a robotic arm control node, a flow meter node, a serial port node, and an interface interaction node, and realizes real-time interaction and synchronization of data and control information through the ROS multi-node communication mechanism.

[0012] The cow identification module includes an RFID reader / writer and a cow data management unit. The RFID reader / writer is installed at the entrance of the milking station and automatically reads the electronic ear tag of the cow when it enters the station. The identified ID number is transmitted to the host computer control unit via a serial port. The cow data management unit is used to record and manage the vital characteristics of the cow, including the number of milkings, milking time, milking volume, and teat status.

[0013] The visual recognition module uses a depth camera to identify and spatially locate the cow's teats. The depth camera is mounted on the end flange of the robotic arm, and the optical axis is tilted upward at an angle of 10° to 15° relative to the normal of the flange to collect RGB-D image data of the cow's udder area.

[0014] The robotic arm motion control module consists of a servo-driven multi-degree-of-freedom robotic arm, with a milk cup execution component and a depth camera installed at the end of the robotic arm to realize point updates and milk cup installation.

[0015] The point update is as follows: converting the position of the cow's teat into the point information under the coordinate system of the robot arm base, specifically including: (1) using hand-eye calibration technology to obtain the rotation matrix R1 and translation vector T1 between the camera coordinate system and the robot arm end coordinate system; using the transformation matrix (R1,T1) to convert the three-dimensional coordinates of the cow's teat under the camera view to the coordinate system of the robot arm end; and then by setting a compensation value, realizing the point calculation of the robot arm end that moves the specified point on the milk tray to the actual teat position, thereby obtaining the target position of the robot arm; (2) based on the difference between the current position information of the cow's teat under the camera view and the final position information of the cow's teat under the camera view when it is located at the specified point, converting the difference, i.e. the actual distance, into the point information under the robot arm coordinate system to obtain the target position of the robot arm; taking the average of the target positions of the robot arm calculated in (1) and (2), and updating the point when four teats are identified;

[0016] Furthermore, the robotic arm motion control module sets two distance thresholds, specifically an emergency stop threshold and a danger distance threshold. When the emergency stop threshold is reached, the robotic arm immediately stops its current motion state, i.e., it is powered down, and enters an adjustment state. According to the dangerous direction of motion, the robotic arm moves in the opposite direction to a safe area. If the danger distance threshold is triggered in the adjustment state, it returns to the starting point from the current position along a safe path. If the danger distance threshold is not triggered, it returns to the previous workflow after moving to a safe distance and cancels the emergency stop state.

[0017] The lower-level relay execution module controls the solenoid valve group, vacuum pump, and pulser through the relay board to realize the execution actions of the milk cup, including tilting, vacuum and pulse control, tilting of the roller brush frame, starting and stopping of the medicine bath spray valve, pipeline switching, and handling and discarding of foreign milk. The relay control board receives digital instructions from the upper-level computer control unit and drives the solenoid valve group to realize the tilting of the milk cup, starting and stopping of the vacuum pump, periodic control of the pulser, tilting of the roller brush frame, starting and stopping of the medicine bath spray valve, flow meter recording, and receiving data from the distance infrared sensor. The lower-level relay execution module periodically collects current, voltage, and signal status and feeds them back to the upper-level computer control unit.

[0018] The sensing and monitoring module includes a TOF distance sensor and a flow meter. The TOF sensor is positioned around the milk tray at the end of the robotic arm, enabling adaptive obstacle avoidance, flow monitoring, and negative pressure safety protection. The flow meter is connected to the milk cup to monitor the real-time flow rate in the milk area. The sensor signal is processed by an A / D conversion module and then sent to the host computer control unit.

[0019] The adaptive intelligent control system of the milking robot also includes a central database; the central database is a continuously updated original database of dairy cows, including various dimensions of data on the condition of the dairy cows themselves and their feeding conditions directly collected from the farm.

[0020] The host computer control unit serves as the central control node of the system. Based on the NVIDIA development platform, it runs the ROS system. The host computer control unit connects to the peripheral communication module via an Ethernet interface. At the same time, it establishes high-speed serial port or CAN bus communication with the cow identification module, vision recognition module, robotic arm motion control module, sensor monitoring module, and lower-level relay execution module to achieve synchronous control of multiple modules.

[0021] The dairy cow identification system uses RFID technology to identify and bind individual cows to data. Specifically, it links a cow's identity with its milk production, health indicators, teat position information, milking frequency, and body length and weight to form a complete data file. Simultaneously, based on the identification results, the system determines whether the cow is being milked for the first time and whether it meets the milking conditions, enabling intelligent decision-making—that is, executing the milking action based on the cow's physical condition. Furthermore, during the first milking, if teat position information is not yet available, identification is performed at the designated location before proceeding with medicated bathing, brush cleaning, and milking. For subsequent milkings, a weighted adjustment is made based on the data file to reduce the time spent on the identification process.

[0022] Furthermore, the dairy cow identification system proposes an open information interconnection architecture. Before a cow enters the pen, its unique identity is confirmed through electronic ear tags, visual recognition, or RFID modules, and a digital file containing body size, teat coordinates, health indicators, milk production records, and feeding information is established in a central database. Before milking, the dairy cow identification system automatically retrieves the file, spatially registers the teat position information with real-time visual recognition results, and achieves individualized positioning and control parameter initialization. By analyzing the file data and real-time sensor information, the dairy cow identification system determines the cow's physiological state and production stage.

[0023] The milking robot motion control system is responsible for converting the identified nipple position information into the robot's workspace coordinate system and planning the robot's motion trajectory to achieve precise positioning and motion control of the robot's end effector at the target point.

[0024] The lower-level relay control system receives instructions from the serial port transceiver node of the control unit and controls the on / off state of the relay, thereby realizing the uprighting and tipping action of the milk cup, and the start and stop control of the vacuum pump and pulser. The lower-level relay control system also monitors the distance to the adjacent side of the robotic arm at high frequency and feeds back to the robotic arm motion control module to realize the adaptive obstacle avoidance function.

[0025] During milking, the robotic arm acts as the actuator, responsible for the movement of the milk tray at the end. This invention establishes an inverse kinematics model of the robotic arm based on its dimensions. Given a specified end-effector position and pose, the angles of each joint and the extension length of the connecting parts can be calculated. This allows the robotic arm controller to achieve precise point-to-point movement. The inverse kinematics model utilizes the DH analysis method, based on the robotic arm joint and arm length information, to calculate the end-effector position of the robotic arm as the rotation angles of each axis joint.

[0026] The adaptive intelligent control system uses a depth camera for nipple localization and identification, and a TOF sensor for real-time obstacle avoidance adjustment, to achieve robust identification and localization of nipple points. The adaptive intelligent control system has visual compensation capabilities under occlusion conditions and can intelligently infer the location of missing targets based on partial information, ensuring recognition accuracy and operational safety.

[0027] Specifically, images containing infrared and depth information are acquired using a depth camera, and foreground and background noise are separated using a fusion algorithm. To address water vapor blur, an image dehazing algorithm based on contrast gradient is used to restore nipple edge details. When the image quality is substandard, the lens wiper is automatically triggered to remove dirt and maintain visual clarity.

[0028] In the identification stage, a convolutional neural network is used to extract nipple features and spatial locations. If all four nipples are identified, the host computer control unit directly guides the robotic arm to align. If there is partial occlusion that results in only three nipples being identified, a geometric template is called. Based on the characteristic that cow nipples are often distributed in an isosceles trapezoidal shape, the location of the fourth nipple is inferred by combining the coordinates of three points and individual historical data, and then output after weight correction.

[0029] If fewer than three points can be identified, the host computer control unit initiates the active exploration mode, controlling the robotic arm to conduct a small search within a safe range, while the depth camera continuously collects data and updates the identification results; once a new nipple is discovered, the search is immediately stopped and the target layout is recalculated, gradually restoring complete information and achieving autonomous vision-motion correction;

[0030] To address the problem that PID control cannot compensate for disturbances caused by vacuum pulsation and sudden changes in milk flow in real time, this invention proposes a dynamic synchronous adaptive control mechanism for the position of the robotic arm and the nipple.

[0031] Specifically, by real-time monitoring of the position of the robotic arm's end effector and the nipple's adsorption state, the robotic arm's motion control and vacuum regulation are dynamically coupled to form a two-variable coordinated control of position and flow rate, so as to maintain adsorption stability and negative pressure balance under unsteady working conditions.

[0032] In terms of robotic arm motion control, the host computer control unit calculates the spatial error between the end of the robotic arm and the teat in real time. For the movement or shaking of the cow during milking, an adaptive gain adjustment mechanism is introduced into the traditional PID control. The PID parameters are dynamically adjusted according to the displacement and vacuum change rate to ensure that the controller maintains the optimal response under different disturbances. Once a change in the cow's posture is detected, the host computer control unit drives the robotic arm to perform micro-compensation movement, synchronously following the dynamic position of the teat to ensure stable contact between the milking cup and the teat.

[0033] To further enhance robustness under nonlinear disturbances, a fuzzy logic regulator is introduced into the host computer control unit. The fuzzy logic regulator automatically determines the stability of the current operating condition based on real-time monitoring of nipple displacement, vacuum fluctuations, and nipple flow rate changes, and switches the control gain and response rate parameters accordingly.

[0034] When any abnormal fluctuation is detected in any vacuum milk suction channel, the host computer control unit immediately triggers the protection logic, reduces the suction force and suspends the movement of the robotic arm. After the flow rate of the milk read by the flow meter returns to a stable level, the host computer control unit drives the robotic arm to automatically resume work, realizing safe self-recovery operation throughout the entire process.

[0035] On the other hand, the present invention also provides an adaptive intelligent control method for a milking robot, implemented through the aforementioned adaptive intelligent control system for a milking robot, comprising the following steps:

[0036] S1. System initialization and standby; After the host computer control unit is started, the robotic arm motion control module is automatically powered on and controls the robotic arm to move to a preset safe "zero position"; At the same time, all milking cup groups perform cup standing operation. At this time, the adaptive intelligent control system enters the standby cycle, continuously listens for external commands, and waits for the "start milking" start command.

[0037] S2. Upon receiving the start command, the system first obtains the cow's identity information through a barcode reader or RFID technology; based on the identification result, it determines whether the cow is a new cow being milked for the first time or an experienced cow being milked again.

[0038] For new cows, the complete initialization process, including visual calibration and milk cup preparation, is initiated, i.e., step S3 is skipped; while for experienced cows, the stored historical data is directly called to enter the core operation process, i.e., step S5 is skipped.

[0039] S3. For new cows, the robotic arm first moves to a fixed shooting position preset according to the working environment, and the depth camera captures and archives images of the cow's udder area; then, the robotic arm moves to a position directly below the cow's abdomen and with the milk tray parallel to the cow, where the depth camera continuously collects images. By fusing the spatial mapping method based on the calibration matrix and the dynamic compensation method based on visual difference, the coordinates are solved and adaptively corrected. The coordinates are calculated and optimized in real time to obtain the coordinate set of the four teats.

[0040] S4. By controlling the solenoid valve group, the four milk cups are sequentially changed from an upright standby state to a tilted posture that is biased towards the camera without obstructing the camera.

[0041] S5. Perform medicated bath operation: The robotic arm positions itself under the four target nipples in sequence according to the calculated coordinates; after reaching each designated position, the host computer control unit sends a command to open the medicated bath spray valve to spray disinfectant on the nipple for several seconds, and then automatically shuts off after completion;

[0042] S6. After the medicated bath is completed, the roller brush cleaning operation is performed; the robotic arm moves again to the preset cleaning target position; at each point, the host computer control unit controls the roller brush bracket to rise and start rotating, while the robotic arm makes up-and-down micro movements in the vertical direction to simulate the manual wiping action and perform a thorough physical cleaning of the nipple; after the operation is completed, the roller brush bracket automatically retracts to prepare for the next operation.

[0043] S7. Entering the cup-attaching and milking stage; the robotic arm carries the milk cups and moves sequentially to the calculated cup-attaching coordinate point, that is, the calculated position of the cup-attaching point in the robotic arm coordinate system relative to the nipple in the robotic arm coordinate system. By executing the rope slack action, the milk cup can be attached to the nipple; after confirming that the milk cup is in place, the vacuum pump and pulsator are started sequentially to establish the negative pressure environment required for milking, and instructions are issued simultaneously to activate all flow meters to start monitoring the milk flow in real time. After discarding the initial milk, the formal milking process begins;

[0044] S8. During milking, the host computer control unit continuously monitors the real-time data returned by the four flow meters and independently judges and makes decisions on the status of each milking zone; when the flow rate is maintained within the preset range, the current parameters are maintained and the operation continues; when the flow rate of a milking zone is detected to be lower than the preset range, the working frequency of the pulsator is increased; once it is determined that the flow rate of a milking zone has disappeared, the cup removal operation is immediately performed for that milking zone, including closing the milk valve, stopping the vacuum and retrieving the milk cup, until all milking zones have been milked;

[0045] S9. After all milking is completed, perform another medicated bath to disinfect the teats. Finally, the robotic arm carrying the milk cup set safely returns to the zero position. The adaptive intelligent control system resets all internal status flags and uploads the time and yield of this operation to the dairy cow data management unit for production analysis.

[0046] The beneficial effects of adopting the above technical solution are as follows:

[0047] This invention provides an adaptive intelligent control system for milking robots, addressing key technical bottlenecks exposed in the application of existing milking robots in Chinese dairy farms. The core of this system lies in resolving issues such as the difficulty of static models adapting to the differences in body size among Chinese dairy cows, leading to high rates of missed milking and teat drop; the susceptibility of single-vision systems to recognition failures under conditions of manure and moisture, causing robotic arm collisions and cow stress; the inability of traditional PID control to compensate for vacuum and flow disturbances in real time, resulting in excessive vacuum fluctuations at the teat end and exacerbating the risk of mastitis; and the closed nature of existing systems, preventing integration into the existing intelligent ecosystem of dairy farms and creating information silos. In contrast, this solution achieves a highly accurate teat positioning and robust operation in complex environments through point-to-point adaptive control, multi-level occlusion compensation and redundant recognition, dynamic coupling and coordinated control of the robotic arm and vacuum system, and an open information interconnection architecture. It also enables data-driven precision management based on individual cow records. This system significantly improves the automation level, reliability, and animal welfare of milking operations, while providing key technical support for intelligent management and control throughout the entire life cycle of large-scale dairy farms. Attached Figure Description

[0048] Figure 1 An adaptive intelligent control system block diagram of a milking robot according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of the overall structure of the milking robot according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the milk tray structure of the milking robot according to an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of the milking robot's workflow according to an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the multi-level occlusion compensation and redundancy recognition system according to an embodiment of the present invention;

[0053] The components are: 1-Four-degree-of-freedom industrial robotic arm, 2-Milk tray, 3-End flange, 4-RFID reader, 5-Fixed frame, 6-Intel RealSense D435i depth camera, 7-Vacuum tubing, 8-Flexible breast pump cup, 9-Milk holder, 10-Nozzle, 11-TOF distance sensor, 12-Roller brush holder, 13-Pneumatic motor, 14-Roller brush. Detailed Implementation

[0054] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0055] Example 1:

[0056] On the one hand, the present invention provides an adaptive intelligent control system for a milking robot, such as Figure 1 As shown, it includes a host computer control unit, a cow identification module, a vision recognition module, a robotic arm motion control module, a sensor monitoring module, and a lower-level relay execution module. Each module operates as an independent node, including a cow number reading node, a cow teat identification node, a robotic arm control node, a flow meter node, a serial port node, and an interface interaction node. Real-time interaction and synchronization of data and control information are achieved through the ROS multi-node communication mechanism.

[0057] By employing a high-precision robotic arm and visual recognition algorithms, the system automatically locates the cow's teats and accurately installs the milk cups, replacing traditional manual milking operations, reducing labor, and improving milking efficiency and hygiene. The device incorporates a multi-sensor fusion system that collects real-time information on the cow's body surface features, posture changes, and udder position. Dynamic compensation and path optimization are then performed through control algorithms, enabling the robotic arm to maintain high robustness and adaptability in unstructured farm environments.

[0058] The cow identification module includes an RFID reader / writer and a cow data management unit. The RFID reader / writer is installed at the entrance of the milking station and automatically reads the electronic ear tag of the cow when it enters the station. The identified ID number is transmitted to the host computer control unit via a serial port. The cow data management unit is used to record and manage the vital characteristics of the cow, including the number of milkings, milking time, milking volume, and teat status.

[0059] The visual recognition module uses an Intel RealSense D435i depth camera 6 to complete the identification and spatial positioning of cow teats. The depth camera is mounted on the flange at the end of the robotic arm, and the optical axis is tilted upward at an angle of 10° to 15° relative to the normal of the flange to ensure the recognition range and accuracy. It collects RGB-D image data of the cow udder area. The module is connected to the host computer through a USB 3.0 interface and runs the camera driver node, image processing node and teat detection node under the ROS framework to realize the complete process from image acquisition to three-dimensional coordinate recognition.

[0060] The robotic arm motion control module consists of a servo-driven multi-degree-of-freedom robotic arm. In this embodiment, a four-degree-of-freedom industrial robotic arm 1 with a four-degree-of-freedom serial structure is used and mounted on a fixed frame 5. A milk cup execution component and a depth camera are installed at the end of the robotic arm to realize point update and milk cup installation.

[0061] The point update is as follows: the position of the cow's teat is converted into the point information under the coordinate system of the robot arm base. In order to ensure real-time identification of the teat and avoid the occurrence of occlusion as much as possible, the D435i depth camera is fixed at the end flange 3 of the robot arm and tilted upward at a certain angle. (1) The rotation matrix R1 and translation vector T1 between the camera coordinate system and the end coordinate system of the robot arm are obtained by using hand-eye calibration technology; the three-dimensional coordinates of the cow's teat under the camera view are converted to the end coordinate system of the robot arm by the transformation matrix (R1,T1); in order to facilitate the realization of robot arm control, combined with the robot arm's own control logic, and by setting the compensation value, the point calculation of the robot arm end which moves the specified point on the milk tray 2 to the actual teat position is realized, thereby obtaining the target position of the robot arm; this technical route has a certain error due to the adjustability of the end flange rotation angle, and the error is reduced to zero as the angle decreases. (2) Based on the difference between the current position information of the cow's teat in the camera view and the final position information of the cow's teat when it is located at a specified point, the difference, i.e., the actual distance, is converted into point information in the robot's coordinate system to obtain the target position of the robot. After testing, a maximum positioning error of 3mm can be achieved in the robot's workspace, but due to the angle error, there is also a difference in the motion accuracy of the edge area. Combining the above two technical routes, the average value of the target position of the robot calculated in (1) and (2) is taken to reduce the error, and the point is updated when four teats are identified. Through the above theory, the adaptive strategy of the present invention can realize the functional requirement of the robot following the teat in real time.

[0062] Furthermore, the robotic arm motion control module sets two distance thresholds, specifically an emergency stop threshold and a danger distance threshold. When the emergency stop threshold is reached, the robotic arm immediately stops its current motion state, i.e., it is powered down, and enters an adjustment state. Based on the dangerous direction of movement, the robotic arm moves in the opposite direction to a safe area. If the danger distance threshold is triggered in the adjustment state, it returns to the starting point from the current position along a safe path. If the danger distance threshold is not triggered, it returns to the previous workflow after moving to a safe distance and cancels the emergency stop state. Through real-time obstacle avoidance and adjustment strategies, the robotic arm can gradually avoid dangerous areas and complete the functional requirement of moving to the designated point.

[0063] The lower-level relay execution module controls the solenoid valve group, vacuum pump, pulser, and pneumatic motor 13 via the relay board to realize the execution actions of the milk cup, including tilting, vacuum and pulse control, tilting of the roller brush holder 12, starting and stopping of the medicine bath spray valve, pipeline switching, foreign milk treatment and discarding, and roller brushing. The relay control board receives digital instructions from the upper-level computer control unit and drives the solenoid valve group to realize the tilting of the milk tray 9 and milk cup, starting and stopping of the vacuum pump, periodic control of the pulser, tilting of the roller brush holder 12, starting and stopping of the medicine bath spray valve, flow meter recording, and receiving data from the distance infrared sensor. The lower-level relay execution module periodically collects current, voltage, and signal status and feeds it back to the upper-level computer control unit to realize safety detection and execution confirmation.

[0064] The sensing and monitoring module includes a Time-of-Flight (TOF) distance sensor and a flow meter. The TOF sensor is arranged around the milk tray 2 at the end of the robotic arm to achieve adaptive obstacle avoidance, flow monitoring, and negative pressure safety protection. In this embodiment, three TOF distance sensors are used, installed on the front and sides of the milk tray respectively, for real-time detection of obstacle distances. The flow meter is connected to the milk cup to monitor the instantaneous flow rate in the milking area. The sensor signals are processed by the A / D conversion module and then sent to the host computer control unit to participate in the adaptive obstacle avoidance and milking control logic.

[0065] In this embodiment, for real-time obstacle avoidance adjustment, three TOF sensors are arranged on the front and two sides of the milk tray at the end of the robotic arm to obtain obstacle distance information in three directions in real time.

[0066] The adaptive intelligent control system of the milking robot also includes a central database; the central database is a continuously updated original database of dairy cows, including various dimensions of data on the condition of the dairy cows themselves and their feeding conditions directly collected from the farm.

[0067] The host computer control unit, serving as the central control node of the system, is based on the NVIDIA development platform. In this embodiment, it uses the NVIDIA Jetson Xavier NX (16GB) platform and runs the ROS (Robot Operating System). This unit runs the Ubuntu 20.04 operating system and has ROS Noetic installed, undertaking upper-level decision-making functions such as visual information processing, path planning, data fusion, and task scheduling. Furthermore, the host computer control unit includes a cow identification system, a milking robot motion control system, and a lower-level relay control system. The host computer control unit connects to the peripheral communication module via an Ethernet interface and establishes high-speed serial or CAN bus communication with the cow identification module, vision recognition module, robot motion control module, sensor monitoring module, and lower-level relay execution module to achieve synchronous control of multiple modules.

[0068] The dairy cow identification system uses RFID technology to achieve individual cow identification and data binding. Specifically, it links the cow's identity with its milk production, health indicators, teat position information, milking frequency, and body length and weight to form a complete data file. Simultaneously, based on the cow identification results, the system determines whether the cow is being milked for the first time and whether it meets the milking conditions, enabling intelligent decision-making. This means that milking is performed based on the cow's physical condition; if the cow's teats are in poor condition, milking is not performed, otherwise it proceeds normally. Furthermore, during the first milking, if teat position information is not yet available, identification is performed at the designated location before medicated bathing, cleaning with a roller brush, and milking. For the second or subsequent milkings, a weighted adjustment is made based on the data file to reduce the time spent on the identification process.

[0069] Furthermore, addressing the issues of closed systems, inability to connect to livestock big data platforms, and information silos, the dairy cow identification system proposes an open information interconnection architecture. Its core control system lies in the mapping between identity data and control parameters. Before a cow enters the pen, its unique identity is confirmed through electronic ear tags, visual recognition, or RFID modules, and a digital file containing body size, teat coordinates, health indicators, milk production records, and feeding information is established in a central database. Before milking, the dairy cow identification system automatically retrieves the file, spatially registering the teat position information with real-time visual recognition results to achieve individualized positioning and control parameter initialization, avoiding redundant calibration and improving efficiency and accuracy. In terms of control logic, a data-driven adaptive milking strategy generation mechanism is introduced. The dairy cow identification system analyzes file data and real-time sensor information to determine the cow's physiological state and production stage. For example, for healthy cows in peak lactation, the system relaxes the negative pressure fluctuation threshold to increase flow rate; while for individuals at risk of mastitis, it automatically lowers the vacuum pulsation frequency and extends the slow-inhalation phase, achieving flexible control that balances yield and cow health.

[0070] The milking robot motion control system is responsible for converting the identified teat position information into the robot's workspace coordinate system and planning the robot's motion trajectory, achieving precise positioning and motion control of the robot's end effector at the target point. This system achieves adaptive adjustment through a real-time feedback mechanism to cope with minor changes in the cow's posture or position.

[0071] The lower-level relay control system receives commands from the serial port transceiver node of the control unit, controlling the on / off state of the relays to realize the uprighting and tipping action of the milk cups, and the start / stop control of the vacuum pump and pulser, ensuring the sequence and safety of the milking process. In addition, the lower-level relay control system also monitors the distance to the adjacent sides of the robotic arm at high frequency, feeding back to the robotic arm motion control module to realize adaptive obstacle avoidance function.

[0072] The ROS system employs an asynchronous, non-blocking mechanism for topic communication. Emergency stop detection messages are transmitted at a frequency of 20Hz, while other information is transmitted at 5Hz, ensuring real-time data exchange. The system uses a distributed parameter server to store key operating parameters (such as vacuum threshold, obstacle avoidance distance, and camera calibration parameters), supporting online modification and saving.

[0073] Upon power-up, the system automatically executes the node initialization process. The startup order of each node is controlled by the roslaunch configuration file to ensure the correct loading of dependencies. The overall system running status is uniformly published for centralized monitoring via the interface.

[0074] The adaptive intelligent control of this system is manifested in a closed loop of "perception-decision-execution-feedback": visual and RFID data are input into the host computer, and control commands are generated through a fusion algorithm. After the robotic arm executes the commands, feedback from flow / distance sensors provides real-time optimization. This dynamic adjustment mechanism enables the system to cope with unstructured environmental disturbances such as cow movement and changes in lighting.

[0075] The adaptive intelligent control system described in this invention has good scalability and modularity, and can be upgraded and expanded according to the scale of the farm and application requirements.

[0076] The adaptive intelligent control system proposed in this invention, through a three-layer collaborative control logic of perception, decision-making and execution, can adaptively adjust control parameters in different working stages, and realize intelligent closed-loop control from individual difference identification to group data optimization.

[0077] In terms of overall design concept, the core innovation of this invention is not limited to the optimization of a single link, but rather constitutes a complete adaptive intelligent control framework through the organic synergy of four subsystems: "point-based adaptive control with dual technical routes integration, multi-level redundant visual recognition, dynamic coupling and coordinated control, and open information interconnection architecture." This framework, driven by real-time data, centered on control feedback, and based on multi-source information fusion, achieves high robustness and adaptive intelligent evolution capabilities for the milking robot in multi-dimensional scenarios such as spatial dynamics, environmental disturbances, and information sharing.

[0078] To address the problem that static models are unable to adapt to differences in cow size and posture, leading to high rates of missed milking and cup loss, this invention proposes an adaptive control strategy that integrates two technical approaches.

[0079] During milking, the robotic arm acts as the actuator, responsible for the movement of the milk tray at the end. This invention establishes an inverse kinematics model of the robotic arm based on its dimensions. Given a specified end-effector position and pose, the angles of each joint and the extension length of the connecting parts can be calculated. This allows the robotic arm controller to achieve precise point-to-point movement. The inverse kinematics model utilizes the DH analysis method, based on the robotic arm joint and arm length information, to calculate the end-effector position of the robotic arm as the rotation angles of each axis joint.

[0080] The serial manipulator involved in this invention has significant advantages such as large working space and strong load capacity. However, during milking, the area under the cow's udder is narrow, and the movement trajectory of the manipulator must accurately complete the movement to the required point while ensuring the safety of the cow.

[0081] To address the problem that single-vision localization is prone to recognition failure under interference from feces, water vapor, etc., this invention proposes a multi-level occlusion compensation and redundant recognition mechanism.

[0082] The adaptive intelligent control system uses a depth camera for nipple localization and identification, and a TOF sensor for real-time obstacle avoidance adjustment. Combined with environmental perception, geometric modeling, and motion cooperative control, it achieves robust identification and localization of nipple points in complex environments. The adaptive intelligent control system has visual compensation capabilities under occlusion conditions and can intelligently infer the location of missing targets based on partial information, ensuring recognition accuracy and operational safety.

[0083] Specifically, images containing infrared and depth information are acquired using a depth camera, and foreground and background noise are separated using a fusion algorithm. To address water vapor blur, an image dehazing algorithm based on contrast gradient is used to restore nipple edge details. When the image quality is substandard, the lens wiper is automatically triggered to remove dirt and maintain visual clarity.

[0084] In the identification stage, a convolutional neural network is used to extract nipple features and spatial locations. If all four nipples are identified, the host computer control unit directly guides the robotic arm to align. If there is partial occlusion that results in only three nipples being identified, a geometric template is called. Based on the characteristic that cow nipples are often distributed in an isosceles trapezoidal shape, the location of the fourth nipple is inferred by combining the coordinates of three points and individual historical data, and then output after weight correction.

[0085] If fewer than three points can be identified, the host computer control unit initiates the active exploration mode, controlling the robotic arm to conduct a small search within a safe range, while the depth camera continuously collects data and updates the identification results; once a new nipple is discovered, the search is immediately stopped and the target layout is recalculated, gradually restoring complete information and achieving autonomous vision-motion correction;

[0086] This invention enables robots to maintain high recognition rates and operational safety in complex environments such as high humidity and occlusion by cleaning and defogging the visual perception layer and performing quality self-inspection, as well as geometric inference and dynamic compensation in the recognition layer.

[0087] To address the problem that PID control cannot compensate for disturbances caused by vacuum pulsation and sudden changes in milk flow in real time, this invention proposes a dynamic synchronous adaptive control mechanism for the position of the robotic arm and the nipple.

[0088] Specifically, by real-time monitoring of the position of the robotic arm's end effector and the nipple's adsorption state, the robotic arm's motion control and vacuum regulation are dynamically coupled to form a two-variable coordinated control of position and flow rate. This maintains adsorption stability and negative pressure balance under unsteady conditions, reduces vacuum fluctuations at the nipple end, and minimizes the risk of breast damage.

[0089] In terms of robotic arm motion control, the host computer control unit calculates the spatial error between the end of the robotic arm and the teat in real time. For the movement or shaking of the cow during milking, an adaptive gain adjustment mechanism is introduced into the traditional PID control. The PID parameters are dynamically adjusted according to the displacement and vacuum change rate to ensure that the controller maintains the optimal response under different disturbances. Once a change in the cow's posture is detected, the host computer control unit drives the robotic arm to perform micro-compensation movement, synchronously following the dynamic position of the teat to ensure stable contact between the milking cup and the teat.

[0090] To further enhance robustness under nonlinear disturbances, a fuzzy logic regulator is introduced into the host computer control unit. Based on real-time monitoring of nipple displacement, vacuum fluctuations, and nipple flow rate changes, the fuzzy logic regulator automatically determines the stability of the current operating condition and switches the control gain and response rate parameters accordingly, thereby achieving high-precision control and avoiding oscillations at different operating stages.

[0091] When any abnormal fluctuation is detected in any vacuum milk suction channel, the host computer control unit immediately triggers protection logic, reducing the suction force and pausing the robotic arm's movement to prevent discomfort or injury to the cows. Once the milk flow rate read by the flow meter stabilizes, the host computer control unit drives the robotic arm to automatically resume operation, achieving safe and self-recovering operation throughout the entire process.

[0092] In this way, milking robots no longer operate in isolation, but are integrated into the smart farm ecosystem, enabling them to dynamically adjust control strategies based on global information and achieve synergistic optimization of production efficiency, animal health, and system energy consumption.

[0093] On the other hand, the present invention also provides an adaptive intelligent control method for a milking robot, implemented through the aforementioned adaptive intelligent control system for a milking robot, comprising the following steps:

[0094] S1. System Initialization and Standby: After the host computer control unit starts, the robotic arm motion control module automatically powers on and controls the robotic arm to move to a preset safe "zero position." Simultaneously, all milking cups perform a cup-standing operation, restoring them from their working state to their initial upright, ready-to-go posture. At this point, the adaptive intelligent control system enters a standby cycle, continuously listening for external commands and waiting for the "start milking" command to initiate the entire automated operation sequence.

[0095] S2. Upon receiving the start command, the system first obtains the cow's identity information through a barcode reader or RFID technology. This step is the logical decision point of the entire process, determining whether the cow is a new cow being milked for the first time or an experienced cow being milked again. This determination will directly determine the subsequent execution path.

[0096] For new cows, the complete initialization process, including visual calibration and milk cup preparation, is initiated, i.e., step S3 is skipped; while for experienced cows, the stored historical data is directly called to enter the core operation, i.e., step S5 is skipped, thereby improving operation efficiency.

[0097] S3. For new cows, the system needs to perform visual calibration to establish a working benchmark. The robotic arm first moves to a fixed image-taking position preset according to the working environment, where a depth camera captures and archives images of the cow's udder area. Subsequently, the robotic arm moves to a position directly below the cow's abdomen with the teat parallel to the cow. The depth camera continuously acquires images at this position. By fusing a spatial mapping method based on the calibration matrix and a dynamic compensation method based on visual differences, the system achieves coordinate solving and adaptive correction of the coordinates. The coordinates of the four teats are calculated and optimized in real time, providing a spatial benchmark for the precise movements of the robotic arm.

[0098] S4. After obtaining the visual calibration data, the system needs to make mechanical preparations for the subsequent preprocessing operations. By controlling the solenoid valve assembly, the four milk cups are sequentially changed from an upright, ready-to-go state to a tilted posture that is biased towards the camera without obstructing it. This key action aims to reserve sufficient and safe mechanical movement space for subsequent medicated bath spraying and roller brush cleaning operations, effectively preventing motion interference between the end effector and the robotic arm body, and ensuring a smooth and error-free process.

[0099] S5. After completing the preliminary preparations, the system enters the core preprocessing stage, first performing the disinfectant bath operation: the robotic arm positions itself under the four target nipples in sequence according to the calculated coordinates; after reaching each designated position, the host computer control unit sends a command to open the disinfectant bath spray valve, spraying disinfectant onto the nipple for several seconds, and then automatically shuts off after completion; this process integrates real-time safety monitoring, if an obstacle avoidance signal is triggered, the system can immediately pause and fine-tune the avoidance, and continue execution after the risk is eliminated, ensuring the continuity and safety of the operation.

[0100] S6. After the medicated bath, the roller brush cleaning operation is performed; the robotic arm moves again to the preset cleaning target position; at each point, the host computer control unit controls the roller brush bracket to rise and start rotating, while the robotic arm makes micro-movements up and down in the vertical direction, simulating manual wiping action, to thoroughly physically clean the teats, further ensuring milking hygiene and effectively stimulating milk production in dairy cows. After the operation is completed, the roller brush bracket automatically retracts, preparing for the next operation;

[0101] S7. Entering the cup-attaching and milking stage; the robotic arm carries the milk cups and moves sequentially to the calculated cup-attaching coordinate point, that is, the calculated position of the cup-attaching point in the robotic arm coordinate system relative to the nipple in the robotic arm coordinate system. By executing the rope slack action, the milk cup can be attached to the nipple; after confirming that the milk cup is in place, the vacuum pump and pulsator are started sequentially to establish the negative pressure environment required for milking, and instructions are issued simultaneously to activate all flow meters to start monitoring the milk flow in real time. After discarding the initial milk, the formal milking process begins;

[0102] S8. During milking, the host computer control unit continuously monitors the real-time data returned by the four flow meters and independently judges and makes decisions regarding the status of each udder. When the flow rate remains within the preset range, the current parameters are maintained and operation continues. When the flow rate of a udder is detected to be lower than the preset range, the operating frequency of the pulsator is increased to attempt to improve milking efficiency. Once it is determined that the flow rate of a udder has disappeared (dry milk), the cup removal operation is immediately performed on that udder, including closing the milk valve, stopping the vacuum, and retrieving the milk cup, thereby maximizing the protection of the cow's health while milking the cow completely. This process achieves completely "on-demand milking" until all udders have been milked.

[0103] S9. After all milking is complete, a second medicated bath is performed to disinfect the teats, forming a complete nursing loop to protect the cow's health. Finally, the robotic arm carrying the milk cup assembly safely returns to the zero position. The adaptive intelligent control system resets all internal status flags and uploads the operation time and yield to the cow data management unit for production analysis. At this point, the system returns to its initial standby state, completing a full automated cycle from activation to completion, ready to await the next cow and begin a new round of operations.

[0104] Example 2:

[0105] refer to Figure 2 and Figure 3 This embodiment provides a visual recognition and robotic arm adaptive control system based on the "Eye-in-Hand" architecture. The system acquires the three-dimensional position information of the cow's teats through a depth camera and combines it with the kinematic model of the robotic arm to achieve high-precision end-effector adaptive positioning control.

[0106] To achieve consistency between visual coordinates and robotic arm coordinates, this embodiment first establishes a spatial calibration model of the vision system. The depth camera 6 is fixedly mounted on the end flange 3 of the robotic arm, with its optical axis forming an adjustable upward tilt angle of 10° to 15° relative to the normal direction of the flange. This allows for full observation of the breast area while preventing the end mechanism from obstructing the field of view.

[0107] To improve the stability and positioning accuracy of identification in unstructured farm environments, this embodiment designs two independent coordinate solution routes and achieves adaptive correction through a fusion strategy.

[0108] First approach: Spatial mapping method based on calibration matrix

[0109] The coordinates of the nipple point identified by the depth camera are first transformed to the coordinate system of the robotic arm's end effector using a calibration matrix. This approach depends on calibration accuracy and the robotic arm's posture, and is suitable for static or low-speed operating environments. Its advantages are direct coordinate transformation and fast calculation speed; its disadvantage is that large changes in the end-effector angle can lead to cumulative errors.

[0110] Second approach: Dynamic compensation method based on visual difference

[0111] This invention compares the three-dimensional position information of the cow's teat in the camera's view coordinate system with its view coordinate data at the target reference position in the current frame, calculates the spatial offset between the two, and converts the offset to the corresponding point information in the robot arm's base coordinate system through an extrinsic parameter matrix, so as to achieve real-time pose compensation and position correction.

[0112] The compensation vector is added to the point calculated from the first route to obtain the final target position, which can reflect the error caused by the slight movement of the cow or the shaking of the robotic arm in real time.

[0113] By fusing the two routes, the final control point P is obtained by weighting the target coordinates P1 and P2 output by the two routes. fin :

[0114] ;

[0115] Where α is the weighting coefficient, the preferred range being 0.6 to 0.8. Experiments have shown that this fusion method can stably control the positioning error within ±3mm.

[0116] like Figure 4 As shown, three TOF distance sensors 11 are installed at the front and sides of the robotic arm's end effector, forming a three-way detection area in the forward and left-right directions. Each sensor outputs obstacle distance values ​​D1, D2, and D3 in real time. The system performs adaptive obstacle avoidance control based on the following dual-threshold strategy:

[0117] First threshold D safe (Preferred to be 200mm): When the detection distance is less than this threshold, the system automatically reduces the robotic arm's movement speed to 50% of its original speed;

[0118] Second threshold D stop（ (Preferred value: 100mm): When the detection distance is lower than this threshold, the robotic arm immediately stops its current action (power off) and enters obstacle avoidance adjustment mode.

[0119] In obstacle avoidance adjustment mode, the system determines the direction of danger based on the location of the trigger sensor and generates a reverse movement command. If a dangerous distance is still detected, it returns to the preset starting point along a safe path; if the distance returns to the safe range, it automatically exits obstacle avoidance mode and resumes the original workflow.

[0120] In addition, the control system is equipped with a "soft limit protection" function, which monitors the joint angle range of the robotic arm in real time to prevent structural interference caused by exceeding the mechanism's limits. All safety events are recorded in the log system and uploaded to the database for maintenance and analysis.

[0121] This embodiment achieves real-time tracking and high-precision positioning of a dynamic target (cow teats) through visual recognition, coordinate calibration, and a dual-route fusion algorithm. Compared to single-route methods, the fusion strategy of this invention significantly reduces the impact of visual drift and attitude error.

[0122] Example 3:

[0123] refer to Figure 5This embodiment provides a multi-level occlusion compensation and redundancy recognition system for robust identification and dynamic positioning of nipple points in complex environments. The system is based on a visual sensor and combines environmental perception, geometric modeling and motion cooperative control. It not only has the ability to compensate for visual occlusion under conditions, but also can intelligently infer the location of missing targets based on partial point information, thereby maintaining overall recognition accuracy and operational safety.

[0124] In terms of control logic, the system first establishes a multimodal visual perception model based on a depth camera. The camera acquires image data containing infrared and depth information in real time, and separates the foreground region from background noise through a fusion algorithm. To address the common water vapor blurring phenomenon in high-humidity environments, the system employs an image dehazing algorithm based on contrast gradients to preprocess the input image, restoring the detailed features of the nipple edges. When the image quality falls below a set threshold, the system automatically triggers the lens maintenance module, which uses a scraper located near the lens to perform surface cleaning to remove fecal matter or water droplets, maintaining the clarity and reliability of the vision system.

[0125] In the visual recognition stage, the system utilizes a convolutional neural network model to extract teat features and identify spatial locations in the preprocessed image. When the detection module identifies all four teat locations, the control system can directly input them as target points for the robotic arm, achieving rapid and accurate alignment of the milking cup. However, in actual working environments, some teats may not be directly identified due to obstruction by the cow's legs, dirt interference, or light reflection. To address this, this invention proposes a geometric inference mechanism based on partial locations. When the system identifies only three teat locations, the control module retrieves geometric template information stored in the database. This template is based on the physiological structural feature that most cow teats are distributed in an isosceles trapezoidal shape. The system uses the coordinates of the three points as input and calculates the similarity between the isosceles constraint and the historical individual model to infer the possible spatial location of the fourth teat. This predicted point is corrected by fusion weights before being input to the controller to ensure that it maintains a balance with the real-time visual results in terms of spatial consistency.

[0126] When fewer than three identifiable points are found, the system abandons reliance on single-vision calculations and initiates an active exploration mode. The control system instructs the robotic arm to perform micro-movements along a safe space near the current identification point, ensuring that the arm's movement covers the potential teat area without colliding with the cow's legs or body. During the scanning process, the vision system continuously collects data and updates the identification results in real time. When a new teat point is detected, the system immediately stops scanning, recalculates the target spatial layout, and gradually restores the complete point information. This dynamic compensation strategy based on vision-motion coupling enables the robot to autonomously explore and self-correct in visually constrained environments.

[0127] Through the aforementioned vision and control collaborative mechanism, this invention achieves two-layer robust protection. The visual perception layer ensures input quality through a scraper, image dehazing, and confidence self-detection; the recognition layer reconstructs missing points through geometric constraints and individual profile data. These two elements form a coupled adaptive control closed loop, enabling the milking robot to maintain high recognition rates and safe operation even in complex farming environments such as high humidity, low light, and partial shading.

[0128] This example addresses the problem of delayed response and excessive vacuum fluctuations in the control system of existing milking robots due to changes in cow posture, sudden changes in milk flow, and vacuum pulsation disturbances during operation. An adaptive synchronous control principle is proposed.

[0129] This control principle dynamically couples the robotic arm motion control with the vacuum regulation process by real-time monitoring of the changes in the position of the robotic arm end and the adsorption state of the nipple, forming a two-variable coordinated control system of position and flow. This maintains adsorption stability and negative pressure balance under unsteady conditions, significantly reduces vacuum fluctuations at the nipple end, and avoids damage to breast tissue and the risk of mastitis.

[0130] In terms of robotic arm motion control, the system calculates the spatial error between the robotic arm's end effector and the teat position in real time. Since cows may move slightly or sway during milking, if the robotic arm remains stationary, the relative displacement between the milking cup and the teat can cause vacuum seal instability, leading to airflow fluctuations. Therefore, this invention introduces an adaptive gain adjustment mechanism based on the traditional PID controller. This mechanism dynamically adjusts the proportional, integral, and derivative parameters according to the teat displacement rate and vacuum change rate, ensuring optimal response under different disturbance intensities. When the system detects a change in the cow's posture, it instantly calculates the teat displacement direction and amplitude and drives the robotic arm's end effector to perform a micro-compensation movement to synchronously follow the teat's dynamic position, ensuring relatively stable contact between the milking cup and the teat.

[0131] To further enhance the system's robustness under nonlinear disturbances, this invention introduces a fuzzy logic controller. The fuzzy logic module automatically determines whether the current operating condition falls into one of three modes: "stable," "slight disturbance," or "severe disturbance," based on real-time monitoring of nipple displacement, vacuum fluctuation amplitude, and nipple flow rate changes. Accordingly, it switches between different control gain and response rate parameters. When the system is in a stable operating condition, the controller maintains a low gain to reduce energy consumption and robotic arm jitter. When a severe disturbance is detected, the system automatically increases control sensitivity, increases position compensation amplitude and vacuum adjustment rate, and achieves rapid suppression. This mechanism ensures that the system maintains high-precision control without oscillations at different operating stages.

[0132] When any abnormal fluctuation is detected in any channel, the control system will immediately trigger protection logic, reducing the suction force and pausing the robotic arm's movement to prevent discomfort or injury to the cows. The control module will automatically resume operation after confirming that the signal has stabilized, achieving fully safe and self-recovering operation.

[0133] Example 4:

[0134] This invention addresses the problems of existing milking robot systems being closed, having a single data interface, and being difficult to interconnect with other intelligent systems on farms. It proposes an intelligent collaborative control principle based on open information interaction and data-driven control.

[0135] The core of the control system lies in the mapping and association between identity data and control parameters. Before a cow enters the pen, the system uniquely confirms its individual identity through electronic ear tags, visual recognition, or RFID reading modules, and establishes a digital file for each cow in a central database. This file includes information such as the cow's body size, teat spatial coordinates, health indicators, historical milk production, mastitis records, and feeding formula. Before milking, the system automatically retrieves the corresponding cow's file data, spatially registering the teat position information with the current visual recognition results, thereby achieving precise individual-level positioning and control parameter initialization. By using the identity recognition-triggered file loading mechanism, the system avoids the need for recalibration for each operation, significantly improving milking efficiency and accuracy.

[0136] In terms of control logic, this invention introduces a data-driven adaptive milking strategy generation mechanism. The control module automatically determines the current physiological state and production stage of the cow by analyzing the correlation between archival data and real-time sensor data. When the system detects that a cow is at its peak lactation period and its mammary gland health indicators are good, the controller automatically relaxes the negative pressure fluctuation threshold to increase the milk flow rate; conversely, if the health data provided by the system indicates that the cow has a mild risk of mastitis, the control system lowers the vacuum pulsation frequency and prolongs the slow-inhalation phase to reduce teat stimulation. Through this data-to-control parameter mapping mechanism, the system can achieve flexible milking control based on individual differences, thereby protecting the health of the cow while ensuring milk production.

[0137] Through the above methods, the milking robot is no longer an independent execution unit, but becomes part of the farm's intelligent ecosystem. Its control strategy can be dynamically adjusted according to global information to achieve synergistic optimization of production efficiency, animal health and system energy consumption.

[0138] Example 5:

[0139] like Figure 4 As shown, this embodiment, based on Embodiments 1 and 2, further provides an automated milking operation process control method and system, realizing autonomous operation of the entire process from identification and positioning to milking completion.

[0140] The system includes a host computer control unit, a cow identification module, a vision recognition module, a robotic arm motion control module, a sensor monitoring module, and a lower-level relay execution module.

[0141] Each module is connected via Ethernet, employing a master-slave communication architecture. The host computer control unit is the master station, responsible for task scheduling and status monitoring; the remaining modules are slave stations, respectively performing tasks such as image acquisition, robotic arm control, fluid circuit management, and safety feedback. The workflow of this embodiment includes nine steps, as detailed below:

[0142] S1: System Initialization. Upon system power-on, the host computer control unit initiates a self-test process. It sequentially checks the robotic arm position status, camera connectivity, and safety sensor signals. If all statuses are normal, the system enters "standby" mode; if an anomaly is detected (such as low vacuum or camera connection failure), the system issues an alarm and stops initialization. During the initialization phase, calibration data from the previous milking and cow identification information are loaded to provide initial parameters for subsequent operations.

[0143] S2: Cow Entry and Identification. After a cow enters the milking pen, the RFID reader 4 at the pen entrance automatically reads its electronic ear tag ID. The system calls the database to compare the cow's file, loading the individual's udder geometry model and historical milking records. If it is the first time identifying an individual, the system enters automatic calibration mode, re-collects the udder's 3D point cloud to create a template; otherwise, it directly calls the stored template for posture matching.

[0144] S3: Nipple Recognition and Localization. The visual recognition module activates depth camera 6 to acquire images of the current cow's udder area. By detecting the nipple target, the system outputs the nipple center pixel coordinates and depth information. The system uses the dual-path coordinate fusion algorithm described in Example 2 to calculate the precise spatial position of the nipple in the robotic arm's base coordinate system. If four nipples are identified, they are sequentially numbered T1 to T4, and their three-dimensional coordinates (P1, P2, P3, P4) are recorded.

[0145] S4. Mechanical preparation. By controlling the rope pulling mechanism 9, the system sequentially changes the four milk cups from an upright, ready-to-go state to a specific tilted posture, reserving sufficient and safe mechanical movement space for subsequent medicated bath spraying and roller brush cleaning operations.

[0146] S5: Nipple Cleaning and Disinfection. Under the command of the central control module, robotic arm 1 controls the nipple cleaning device, i.e., the spray nozzle 10, to move to the first target nipple T1. The spray nozzle built into the cleaning device is controlled by a valve and sprays the cleaning solution for 1-2 seconds. The system sequentially completes the cleaning process for four nipples, T1 to T4.

[0147] S6: Nipple Roller Brush and Stimulation. After cleaning, control the flexible roller brush head 14 on the roller brush holder 12 to contact the nipple surface and perform short-term massage stimulation in sequence, lasting about 5 seconds for each nipple. This process can promote milk flow and improve milking efficiency. The roller brush speed is preferably 300rpm to 500rpm to ensure safety and comfort.

[0148] S7: Automatic docking of the breast pump cups. After cleaning and stimulation, the robotic arm end switches to the milking execution mechanism, which includes four flexible breast pump cups 8 and vacuum tubing 7. Based on the spatial coordinates of each nipple, the robotic arm uses the adaptive trajectory planning algorithm from Example 2 to move the breast pump cups along the optimal path to a position 20mm below the target point. It then slowly rises along the Z-axis at a speed of 5mm / s, finally completing the suction.

[0149] S8: Milking and Milk Volume Monitoring. Once all milking cups have stably absorbed and discarded the initial milk, the milking process begins. The milking pump and milk flow sensors collect real-time flow data Q1~Q4 from each teat and calculate the total flow Q. total When the milk flow rate is detected to be less than 0.05 L / min for 10 consecutive seconds, the system determines that milking of that nipple has ended and automatically releases the vacuum in the corresponding milking cup. All milking process data (flow rate curve, time) is uploaded to the database for subsequent health analysis.

[0150] S9: Teat Cup Removal and Postpartum Disinfection. After milking, the robotic arm sequentially removes the four teat cups and switches to nozzle 10. Nozzle 10 automatically sprays a disinfectant solution, evenly covering the teat surface for 3-5 seconds. After this step, the cow can safely leave the pen. After milking, the robotic arm automatically returns to its initial safe position. The central control module records all operational data for this milking session, including recognized images, trajectory path, milking duration, and milk volume. The system automatically generates a health assessment report based on historical data. If abnormal milk flow is detected on one side, a maintenance reminder will be issued in the background.

[0151] This embodiment organically combines visual recognition, robotic arm trajectory planning, vacuum control, and disinfection management modules to construct an integrated closed-loop process of "recognition—cleaning—stimulation—adsorption—milking—disinfection".

[0152] Compared with traditional semi-automatic milking systems, this system achieves:

[0153] The entire process is unmanned.

[0154] Independent control and status monitoring of each nipple;

[0155] Real-time data upload and health analysis;

[0156] Adaptive position compensation and safe obstacle avoidance.

[0157] Therefore, this embodiment significantly improves milking efficiency and animal welfare, providing a highly reliable automated technology foundation for future intelligent farms.

[0158] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0159] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0160] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the methods disclosed herein and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. An adaptive intelligent control system for a milking robot, characterized in that, It includes a host computer control unit, a cow identification module, a vision recognition module, a robotic arm motion control module, a sensor monitoring module, and a lower-level relay execution module. Each module operates as an independent node, including a cow number reading node, a cow teat identification node, a robotic arm control node, a flow meter node, a serial port node, and a user interface interaction node. Real-time interaction and synchronization of data and control information are achieved through the ROS multi-node communication mechanism. The cow identification module includes an RFID reader / writer and a cow data management unit. The RFID reader / writer is installed at the entrance of the milking station and automatically reads the electronic ear tag of the cow when it enters the station. The identified ID number is transmitted to the host computer control unit via a serial port. The cow data management unit is used to record and manage the vital characteristics of the cow, including the number of milkings, milking time, milking volume, and teat status. The visual recognition module uses a depth camera to identify and spatially locate the cow's teats. The depth camera is mounted on the end flange of the robotic arm, and the optical axis is tilted upward at an angle of 10° to 15° relative to the normal of the flange to collect RGB-D image data of the cow's udder area. The robotic arm motion control module consists of a servo-driven multi-degree-of-freedom robotic arm, with a milk cup execution component and a depth camera installed at the end of the robotic arm to realize point updates and milk cup installation. The lower-level relay execution module controls the solenoid valve group, vacuum pump, and pulsator through the relay board to realize the execution actions of the milk cup, including tilting, vacuum and pulsation control, tilting of the roller brush frame, starting and stopping of the medicine bath spray valve, pipeline switching, and handling and discarding of foreign milk; the relay control board receives digital instructions from the upper computer control unit and drives the solenoid valve group to realize the tilting of the milk cup, starting and stopping of the vacuum pump, periodic control of the pulsator, tilting of the roller brush frame, starting and stopping of the medicine bath spray valve, recording of flow meter data, and receiving data from the distance infrared sensor; the lower-level relay execution module periodically collects current, voltage, and signal status and feeds them back to the upper computer control unit; The sensing and monitoring module includes a TOF distance sensor and a flow meter. The TOF sensor is arranged around the milk tray at the end of the robotic arm to achieve adaptive obstacle avoidance, flow monitoring and negative pressure safety protection. The flow meter is connected to the milk cup to monitor the real-time flow in the milk area. The sensor signal is processed by the A / D conversion module and then sent to the host computer control unit. The host computer control unit serves as the central control node of the system. Based on the NVIDIA development platform, it runs the ROS system. The host computer control unit connects to the peripheral communication module via an Ethernet interface, and establishes high-speed serial port or CAN bus communication with the cow identification module, vision recognition module, robotic arm motion control module, sensor monitoring module, and lower-level relay execution module to achieve synchronous control of multiple modules.

2. The adaptive intelligent control system for a milking robot according to claim 1, characterized in that, The point update is as follows: converting the position of the cow's teat into the point information under the coordinate system of the robot arm base, specifically including: (1) using hand-eye calibration technology to obtain the rotation matrix R1 and translation vector T1 between the camera coordinate system and the robot arm end coordinate system; using the transformation matrix (R1,T1) to convert the three-dimensional coordinates of the cow's teat under the camera view to the coordinate system of the robot arm end; and then by setting a compensation value, realizing the point calculation of the robot arm end that moves the specified point on the milk tray to the actual teat position, thereby obtaining the target position of the robot arm; (2) based on the difference between the current position information of the cow's teat under the camera view and the final position information of the cow's teat under the camera view when it is located at the specified point, converting the difference, i.e. the actual distance, into the point information under the robot arm coordinate system to obtain the target position of the robot arm; taking the average of the target positions of the robot arm calculated in (1) and (2), and updating the point when four teats are identified.

3. The adaptive intelligent control system for a milking robot according to claim 2, characterized in that, The robotic arm motion control module is set with two distance thresholds, specifically an emergency stop threshold and a danger distance threshold. When the emergency stop threshold is reached, the robotic arm immediately stops its current motion state, i.e., it is powered off and enters an adjustment state. Based on the dangerous direction of motion, the robotic arm moves in the opposite direction to a safe area. If the danger distance threshold is triggered during the adjustment state, the user will return to the starting point from the current position along a safe path. If the danger distance threshold is not triggered, the system will return to the previous workflow after moving to a safe distance and cancel the emergency stop status.

4. The adaptive intelligent control system for a milking robot according to claim 3, characterized in that, The adaptive intelligent control system of the milking robot also includes a central database; the central database is a continuously updated original database of dairy cows, including various dimensions of data on the condition of the dairy cows and their feeding situation directly collected from the farm.

5. The adaptive intelligent control system for a milking robot according to claim 4, characterized in that, The dairy cow identification system uses RFID technology to achieve individual cow identification and data binding. Specifically, it links the cow's identity with its milk production, health indicators, teat position information, milking frequency, and body length and weight to form a complete data file. Simultaneously, based on the cow identification results, the system determines whether the cow is being milked for the first time and whether it meets the milking conditions, enabling intelligent decision-making—that is, executing the milking action based on the cow's physical condition. Furthermore, during the first milking, if teat position information is not yet available, identification is performed at the designated location before proceeding with medicated bathing, brush cleaning, and milking. For subsequent milkings, a weighted adjustment is made based on the data file to reduce the time spent on the identification process.

6. The adaptive intelligent control system for a milking robot according to claim 5, characterized in that, The dairy cow identification system proposes an open information interconnection architecture. Before a dairy cow enters the pen, its unique identity is confirmed through electronic ear tags, visual recognition, or RFID modules, and a digital file containing body size, teat coordinates, health indicators, milk production records, and feeding information is established in a central database. Before milking, the dairy cow identification system automatically retrieves the file, spatially registers the teat position information with the real-time visual recognition results, and achieves individualized positioning and control parameter initialization. The dairy cow identification system analyzes the file data and real-time sensor information to determine the physiological state and production stage of the dairy cow. The milking robot motion control system is responsible for converting the identified nipple position information into the robot's workspace coordinate system and planning the robot's motion trajectory to achieve precise positioning and motion control of the robot's end effector at the target point. The lower-level relay control system receives instructions from the serial port transceiver node of the control unit and controls the on / off state of the relay, thereby realizing the uprighting and tipping action of the milk cup, the start and stop control of the vacuum pump and the pulser. The lower-level relay control system also monitors the distance to the adjacent side of the robotic arm at high frequency and feeds back to the robotic arm motion control module to realize the adaptive obstacle avoidance function. During milking, the robotic arm acts as the actuator, responsible for the movement of the milk tray at the end. This invention establishes an inverse kinematics model of the robotic arm based on its relevant dimensions. Given a specified end-effector position and pose, the angles of each joint and the extension length of the connecting parts can be calculated. Then, the robotic arm controller can achieve the precise movement of the robotic arm to the target point. The inverse kinematics model uses the DH analysis method based on the joint and arm length information of the robotic arm to calculate the end-effector position of the robotic arm as the rotation angle of each axis joint.

7. The adaptive intelligent control system for a milking robot according to claim 6, characterized in that, The adaptive intelligent control system uses a depth camera for nipple localization and identification, and a TOF sensor for real-time obstacle avoidance adjustment, to achieve robust identification and localization of nipple points. The adaptive intelligent control system has visual compensation capabilities under occlusion conditions and can intelligently infer the location of missing targets based on partial information, ensuring recognition accuracy and operational safety. Specifically, images containing infrared and depth information are acquired using a depth camera, and foreground and background noise are separated using a fusion algorithm. To address water vapor blur, an image dehazing algorithm based on contrast gradient is used to restore nipple edge details. When the image quality is substandard, the lens wiper is automatically triggered to remove dirt and maintain visual clarity. In the identification stage, a convolutional neural network is used to extract nipple features and spatial locations. If all four nipples are identified, the host computer control unit directly guides the robotic arm to align. If there is partial occlusion that results in only three nipples being identified, a geometric template is called. Based on the characteristic that cow nipples are often distributed in an isosceles trapezoidal shape, the location of the fourth nipple is inferred by combining the coordinates of three points and individual historical data, and then output after weight correction. If fewer than three points can be identified, the host computer control unit initiates an active exploration mode, controlling the robotic arm to conduct a small-scale search within a safe range. The depth camera continuously collects data and updates the identification results. Once a new nipple is discovered, the search is immediately stopped and the target layout is recalculated to gradually restore complete information and achieve autonomous vision-motion correction.

8. The adaptive intelligent control system for a milking robot according to claim 7, characterized in that, To address the issue that PID control cannot compensate for disturbances caused by vacuum pulsation and sudden changes in milk flow in real time, a dynamic synchronous adaptive control mechanism for the position of the robotic arm and the nipple is proposed. Specifically, by real-time monitoring of the position of the robotic arm's end effector and the nipple's adsorption state, the robotic arm's motion control and vacuum regulation are dynamically coupled to form a two-variable coordinated control of position and flow rate, so as to maintain adsorption stability and negative pressure balance under unsteady working conditions. In terms of robotic arm motion control, the host computer control unit calculates the spatial error between the end of the robotic arm and the teat in real time. To address the movement or swaying of the cow during milking, an adaptive gain adjustment mechanism is introduced into the traditional PID control. The PID parameters are dynamically adjusted according to the displacement and vacuum change rate to ensure that the controller maintains the optimal response under different disturbances. Once a change in the cow's posture is detected, the host computer control unit drives the robotic arm to perform micro-compensation movement, synchronously following the dynamic position of the teat to ensure stable contact between the milking cup and the teat.

9. The adaptive intelligent control system for a milking robot according to claim 8, characterized in that, The host computer control unit incorporates a fuzzy logic regulator; the fuzzy logic regulator automatically determines the stability of the current working condition based on the real-time monitored nipple displacement, vacuum fluctuation and milk flow rate changes, and switches the control gain and response rate parameters accordingly. When any abnormal fluctuation is detected in any vacuum milk suction channel, the host computer control unit immediately triggers the protection logic, reduces the suction force and suspends the movement of the robotic arm. After the flow rate of the milk read by the flow meter returns to a stable level, the host computer control unit drives the robotic arm to automatically resume work, realizing safe self-recovery operation throughout the entire process.

10. An adaptive intelligent control method for a milking robot, implemented through the adaptive intelligent control system for a milking robot as described in claim 1, characterized in that, Includes the following steps: S1. System initialization and standby; After the host computer control unit is started, the robotic arm motion control module is automatically powered on and controls the robotic arm to move to a preset safe "zero position"; At the same time, all milking cup groups perform cup standing operation. At this time, the adaptive intelligent control system enters the standby cycle, continuously listens for external commands, and waits for the "start milking" start command. S2. Upon receiving the start command, the system first obtains the cow's identity information through a barcode reader or RFID technology; based on the identification result, it determines whether the cow is a new cow being milked for the first time or an experienced cow being milked again. For new cows, the complete initialization process, including visual calibration and milk cup preparation, is initiated, i.e., step S3 is skipped; while for experienced cows, the stored historical data is directly called to enter the core operation process, i.e., step S5 is skipped. S3. For new cows, the robotic arm first moves to a fixed shooting position preset according to the working environment, and the depth camera captures and archives images of the cow's udder area; then, the robotic arm moves to a position directly below the cow's abdomen and with the milk tray parallel to the cow, where the depth camera continuously collects images. By fusing the spatial mapping method based on the calibration matrix and the dynamic compensation method based on visual difference, the coordinates are solved and adaptively corrected. The coordinates are calculated and optimized in real time to obtain the coordinate set of the four teats. S4. By controlling the solenoid valve group, the four milk cups are sequentially changed from an upright standby state to a tilted posture that is biased towards the camera without obstructing the camera. S5. Perform medicated bath operation: The robotic arm positions itself under the four target nipples in sequence according to the calculated coordinates; after reaching each designated position, the host computer control unit sends a command to open the medicated bath spray valve to spray disinfectant on the nipple for several seconds, and then automatically shuts off after completion; S6. After the medicated bath is completed, the roller brush cleaning operation is performed; the robotic arm moves again to the preset cleaning target position; at each point, the host computer control unit controls the roller brush bracket to rise and start rotating, while the robotic arm makes up-and-down micro movements in the vertical direction to simulate the manual wiping action and perform a thorough physical cleaning of the nipple; after the operation is completed, the roller brush bracket automatically retracts to prepare for the next operation. S7. Entering the cup-attaching and milking stage; the robotic arm carries the milk cups and moves sequentially to the calculated cup-attaching coordinate point, that is, the calculated position of the cup-attaching point in the robotic arm coordinate system relative to the nipple in the robotic arm coordinate system. By executing the rope slack action, the milk cup can be attached to the nipple; after confirming that the milk cup is in place, the vacuum pump and pulsator are started sequentially to establish the negative pressure environment required for milking, and instructions are issued simultaneously to activate all flow meters to start monitoring the milk flow in real time. After discarding the initial milk, the formal milking process begins; S8. During milking, the host computer control unit continuously monitors the real-time data returned by the four flow meters and independently judges and makes decisions on the status of each milking zone; when the flow rate is maintained within the preset range, the current parameters are maintained and the operation continues; when the flow rate of a milking zone is detected to be lower than the preset range, the working frequency of the pulsator is increased; once it is determined that the flow rate of a milking zone has disappeared, the cup removal operation is immediately performed for that milking zone, including closing the milk valve, stopping the vacuum and retrieving the milk cup, until all milking zones have been milked; S9. After all milking is completed, perform another medicated bath to disinfect the teats. Finally, the robotic arm carrying the milk cup set safely returns to the zero position. The adaptive intelligent control system resets all internal status flags and uploads the time and yield of this operation to the dairy cow data management unit for production analysis.

Images