Livestock farm inspection robot autonomous inspection system and method based on reinforcement learning

Abstract

The invention discloses a livestock farm inspection robot autonomous inspection system and method based on reinforcement learning, and relates to the technical field of livestock health management. Comprising the following steps: constructing a plurality of farm environment maps by an inspection robot, dividing a virtual inspectable area, obtaining environment data, receiving real-time position information of a livestock positioning collar, obtaining historical position data of livestock, establishing a trajectory analysis model, and obtaining livestock trajectory analysis data to generate a trajectory prediction model; a livestock activity area in a specific time period is predicted, a to-be-inspected area is determined based on the livestock activity area and the real-time position information, and the inspection robot determines and constructs a learning reinforcement model to optimize an inspection path according to the number, distance and time of livestock in the to-be-inspected area and detects the physical condition of the livestock. And the livestock health condition is determined based on the livestock physical condition and the livestock track analysis data, and the information is uploaded to the cloud server. According to the invention, efficient, fine and intelligent farm management can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of livestock health management, and in particular to an autonomous inspection system and method of a farm inspection robot based on reinforcement learning. Background Art

[0002] With the rapid development of science and technology such as the Internet of Things, cloud computing, big data, artificial intelligence, and robots, digital and intelligent transformation has become an inevitable trend in the development of modern animal husbandry. In modern farms, efficient inspections and real-time anomaly detection are essential to ensure the health of livestock and improve the level of intelligent farming. Traditional manual inspections are time-consuming and labor-intensive, and there is a risk of missed inspections. At present, farms basically monitor livestock through video surveillance cameras, which requires the installation of a large number of cameras, and there are still unavoidable monitoring blind spots. They need to rely on manual monitoring, which has high labor costs and cannot interact and intervene with individual livestock in real time, limiting the application effect of video monitoring in dynamic and complex breeding environments. In contrast, mobile inspection robots can monitor and respond in real time and flexibly, which can not only significantly improve the coverage and efficiency of inspections, but also realize real-time monitoring of the environment and individual livestock.

[0003] At present, farm inspection robots are in a stage of rapid development. Research mainly focuses on improving inspection efficiency and environmental adaptability. At this stage, most of the farm inspection robots in existing research are track-based, running along preset tracks, which can achieve 24-hour efficient and uninterrupted inspections to ensure the comprehensiveness and continuity of monitoring. However, track-based inspection robots have disadvantages such as fixed paths, poor flexibility, complex installation, and difficult maintenance. They are difficult to adapt to environmental changes and temporary inspection needs, and the initial installation and subsequent maintenance workload are large and costly. In contrast, inspection robots with autonomous navigation capabilities can flexibly respond to environmental changes and achieve more efficient and comprehensive inspection coverage.

[0004] Therefore, it is an urgent problem for technical personnel in this field to propose an autonomous inspection system and method for farm inspection robots based on reinforcement learning to solve the problems existing in the prior art. Summary of the invention

[0005] In view of this, the present invention provides an autonomous inspection system and method for a farm inspection robot based on reinforcement learning, which helps to achieve efficient, precise and intelligent farm management.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A reinforcement learning-based autonomous inspection system for farm inspection robots, comprising a signal-connected inspection robot, a positioning collar and a cloud server.

[0008] A method for autonomous inspection of a farm inspection robot based on reinforcement learning for executing an autonomous inspection system of a farm inspection robot based on reinforcement learning comprises the following steps:

[0009] S1. The inspection robot constructs multiple farm environment maps, divides virtual inspection areas and obtains environmental data;

[0010] S2, the inspection robot receives the real-time location information of the livestock positioning collar, obtains the livestock's historical location data, establishes a trajectory analysis model, and obtains livestock trajectory analysis data;

[0011] S3, establishing a trajectory prediction model based on livestock trajectory analysis data and environmental data to predict livestock activity areas in a specific time period;

[0012] S4. Determine the area to be inspected based on the livestock activity area and real-time location information;

[0013] S5. The inspection robot determines to build a learning reinforcement model to optimize the inspection path according to the number of livestock, distance and time in the inspection area, and detects the physical condition of the livestock;

[0014] S6. Determine the health status of the livestock based on the livestock physical condition and livestock trajectory analysis data, and upload the information to the cloud server.

[0015] In the above method, optionally, the livestock trajectory analysis data in S2 includes:

[0016] S21. Use LSTM to learn and extract the historical trajectory of livestock from the time series position data to obtain the trajectory characteristics of individual livestock in the time series;

[0017] S22. Analyze the distance and speed between different time points to obtain the movement characteristics of livestock.

[0018] The above method, optionally, the livestock trajectory prediction in US S3 includes:

[0019] S31, deploying multiple light sensors in different areas of the breeding house to obtain light data corresponding to the sensors;

[0020] S32, constructing a lighting distribution model for the entire breeding house according to the lighting data, and dividing different lighting areas based on the model data;

[0021] S33. Based on the historical data of light and individual livestock trajectories, a multivariate regression model is used to establish a prediction model between livestock behavior and light, and dynamically predict the activity range of livestock in a specific time period.

[0022] In the above method, optionally, optimizing the inspection path in S5 includes:

[0023] S51, using the Perceiver model to extract a unified feature representation and obtain data encoding;

[0024] S52, mapping the data encoding to the same dimension;

[0025] S53, based on the self-attention mechanism, weighted fusion of the mapped data encoding is performed to generate a comprehensive feature representation;

[0026] S54, repeat step S53 to obtain multi-level features;

[0027] S55. Optimize inspection sequence based on reinforcement learning model.

[0028] In the above method, optionally, the reinforcement learning model includes determining the state representation, the action representation and the reward function, and continuously optimizing the inspection path based on the proximal strategy optimization.

[0029] It can be seen from the above technical solution that compared with the prior art, the present invention provides an autonomous inspection system and method of a farm inspection robot based on reinforcement learning, which has the following beneficial effects: the present invention effectively reduces labor costs while also improving inspection efficiency, providing data-driven and efficient decision-making support for livestock health management, and helping to achieve efficient, precise and intelligent farm management. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0031] Figure 1 The present invention discloses a flow chart of an autonomous inspection method of a farm inspection robot based on reinforcement learning. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Reference Figure 1 As shown, the present invention discloses an autonomous inspection system of a farm inspection robot based on reinforcement learning, including a signal-connected inspection robot, a positioning collar and a cloud server.

[0034] Furthermore, the inspection robot specifically includes a bionic quadruped robot dog equipped with a binocular camera, an inertial measurement unit (IMU), an infrared camera, a temperature and humidity sensor, a wireless communication module, an Internet of Things module, as well as a high-performance processor, an edge computing device and a readable storage medium.

[0035] A method for autonomous inspection of a farm inspection robot based on reinforcement learning for executing an autonomous inspection system of a farm inspection robot based on reinforcement learning comprises the following steps:

[0036] S1. The inspection robot constructs multiple farm environment maps, divides virtual inspection areas and obtains environmental data;

[0037] S2, the inspection robot receives the real-time location information of the livestock positioning collar, obtains the livestock's historical location data, establishes a trajectory analysis model, and obtains livestock trajectory analysis data;

[0038] S3, establishing a trajectory prediction model based on livestock trajectory analysis data and environmental data to predict livestock activity areas in a specific time period;

[0039] S4. Determine the area to be inspected based on the livestock activity area and real-time location information;

[0040] S5. The inspection robot determines to build a learning reinforcement model to optimize the inspection path according to the number of livestock, distance and time in the inspection area, and detects the physical condition of the livestock;

[0041] S6. Determine the health status of the livestock based on the livestock physical condition and livestock trajectory analysis data, and upload the information to the cloud server.

[0042] Furthermore, S1 includes: the inspection robot builds a farm environment map based on the simultaneous localization and mapping (SLAM) algorithm. Considering that the farm environment usually includes semi-indoor and open areas, with rich features and obvious structures, and there are dynamic obstacles such as stacking cars, the feature point-based visual ORB-SLAM3 algorithm with good stability, high robustness and good real-time performance is selected. The algorithm supports monocular, binocular, RGB-D cameras, and monocular and binocular visual inertial navigation, and can run stably and in real time in large, medium and small, indoor and outdoor environments. Due to the limited field of view (FOV) of the camera carried by the inspection robot, it is impossible to observe all livestock individuals in the breeding house when inspecting in the peripheral feasible channels. Therefore, it is necessary to divide a virtual inspectable area at the edge of the breeding house based on the robot's field of view and the boundary of the breeding house. The horizontal viewing angle of the binocular camera carried by the inspection robot is θ h , the farthest observation distance is D max .

[0043] Assume that the robot moves along the passage parallel to the edge of the breeding house, looks sideways at the breeding house, and maintains a relatively fixed safety distance d from the railing for inspection when there are no obstacles. At this time, the camera's horizontal field of view angle θ h The range projected onto the interior of the single-sided breeding house represents the horizontal width W of the edge area of ​​the breeding house that the camera can observe at a distance d. inspect , which can be specifically expressed as:

[0044]

[0045] Through the SLAM algorithm, the robot can obtain the initial global map of the farm after completing a round of inspection, obtain the geometric position of the edge railings of each breeding house in the global coordinate system, and describe it as a line segment with a starting point (x1, y1), an end point (x2, y2), and a slope of Then, a virtual inspection area is divided based on the robot's field of view near the edge of the farm house. If the farm house is on the right side of the inspection robot's forward direction, the upper left boundary point of the virtual inspection area is (x1, y1), and the upper right boundary point is The lower left boundary point is (x2,y2), and the lower right boundary point is A rectangular virtual inspection area V can be defined by these four boundary points. inspection .

[0046] Furthermore, considering the inspection needs of farms and the cost of robot systems, this application selects binocular cameras and inertial measurement unit (IMU) sensors for data collection for environmental mapping and subsequent livestock health monitoring. Based on the binocular-inertial mode of the ORB-SLAM3 algorithm, the robot uses the binocular camera and inertial measurement unit (IMU) carried by the robot to perceive the environment, extract and match features, and construct a point cloud map of the farm, representing important geometric structures in the environment, such as walls, obstacles, and fence boundaries of the breeding house.

[0047] Furthermore, the real-time location information in S2 includes: the positioning collar carried by each livestock individual has a built-in GPS module, which can obtain the individual's precise geographic location by receiving satellite signals. The GPS module then regularly sends near-real-time location information to the inspection robot through the communication module. In order to reduce power consumption and extend the use time, a suitable wake-up time interval (every 1 minute or every 3 minutes) is set for the GPS module in the collar to obtain the near-real-time geographic location of the livestock individual c in the i-th breeding house. The information is packaged and transmitted to the LoRa communication module, which then uses the communication module to send the positioning data to the inspection robot system. The system receives this location information in real time and dynamically monitors whether individual livestock enter the virtual inspection area V. inspection , the expression is:

[0048] if(x c ,y c )∈V inspection ;

[0049] When a new livestock is detected entering the virtual inspection area and has not yet been inspected, the system increases the count of individuals to be inspected:

[0050]

[0051] in, is the number of livestock to be inspected in the virtual inspectable area of ​​the current i-th breeding house. When the number reaches the preset threshold, the system automatically triggers a reminder, marks the area as inspectable, and instructs the inspection robot to inspect the house. To optimize battery life, the GPS and LoRa modules in the collar enter low-power sleep mode after completing data collection and transmission until the next time period.

[0052] Furthermore, the livestock trajectory analysis data in S2 includes:

[0053] S21. Use LSTM to learn and extract the historical trajectory of livestock from the time series position data to obtain the trajectory characteristics of individual livestock in the time series;

[0054] S22. Analyze the distance and speed between different time points to obtain the movement characteristics of livestock.

[0055] Specifically, the Long Short-Term Memory (LSTM) network is a special type of Recurrent Neural Network (RNN) that is good at processing time series data and can capture and learn long-term dependencies.

[0056] The location information of livestock individual c in the i-th breeding house at different times t It is a time series data, collecting a certain time window t1, t2, ···, t n The position information in the time interval is used as the input of LSTM, and the position information of each time point is represented as a vector

[0057] Constructing time series input: Assume there are T time steps, each of which has the location data of livestock Combine these position information in time order to get the input sequence The input at each time step is the livestock c at time t k The position coordinates of constitute the input time series.

[0058] The LSTM unit contains three main gating mechanisms: forget gate, input gate, and output gate. These gating mechanisms control the flow of information through the output of the Sigmoid function. The input LSTM network processes the position sequence through the above forget gate, input gate, output gate and memory unit state to extract the time-dependent features of the trajectory.

[0059] Furthermore, the forget gate determines how the memory of the previous time step affects the current state:

[0060]

[0061] Among them, f t is the output of the forget gate, σ is the Sigmoid function, W f is the weight matrix, b f is the bias vector, h t-1 is the hidden state at the previous moment, s t is the input vector at the current moment;

[0062] The input gate controls the effect of the current input on the memory cell:

[0063]

[0064] Among them, i t is the output of the input gate, is a candidate memory cell, W i and W c is the weight matrix, b i and b c is the bias vector.

[0065] Memory cell state c t renew:

[0066]

[0067] Among them, c t is the current cell state, c t-1 is the cell state at the previous moment, ⊙ represents the element-level multiplication operation;

[0068] The output gate controls the output of the memory cell:

[0069]

[0070] Among them, t is the output of the output gate, W o is the weight matrix, bo is the bias vector;

[0071] Hide status updates:

[0072] h t =o t ⊙tanh(c t );

[0073] LSTM outputs the final hidden state h T Extracted the trajectory characteristics of livestock individuals in time series Can be abbreviated as

[0074] The movement characteristics of livestock can be obtained by analyzing the distance and speed between different time points. k and t k+1 The moving distance between them is expressed as Euclidean distance:

[0075]

[0076] Let the time interval be Δt k =t k+1 -t k , then the speed of the individual is:

[0077]

[0078] By analyzing the speed, we can understand how fast the livestock moves and whether they have abnormal behaviors, such as being still for a long time or moving quickly. Visualizing the historical location data and trajectory characteristics on a two-dimensional map of the farm can intuitively display the activity trajectory of the livestock. If the speed of the livestock is close to zero or the moving distance is very short within a certain period of time, it can be determined that the livestock stays at a certain location. Through the analysis of the stay point, important areas such as drinking water points or habitats can be determined.

[0079] Furthermore, livestock trajectory prediction in S3 includes:

[0080] S31, deploying multiple light sensors in different areas of the breeding house to obtain light data corresponding to the sensors;

[0081] S32, constructing a lighting distribution model for the entire breeding house according to the lighting data, and dividing different lighting areas based on the model data;

[0082] S33. Based on the historical data of light and individual livestock trajectories, a multivariate regression model is used to establish a prediction model between livestock behavior and light, and dynamically predict the activity range of livestock in a specific time period.

[0083] Specifically, for (semi) open-air breeding houses, multiple light sensors can be deployed in different areas of the breeding house in combination with the spatial layout, changes in light sources (natural light and artificial light), and livestock activity areas to capture changes in the spatial distribution of light in the house. The sensor is installed above the main livestock activity area to ensure that the light conditions at the height of the individual livestock can be accurately measured. It is usually best to install it at a height of 2-3 meters from the ground. Each light sensor regularly collects light intensity data. The light intensity measured by the sth sensor in the ith breeding house at time t is

[0084] The illumination data of each sensor is used to construct the illumination distribution model of the entire breeding house. The illumination field L(t,x,y) represents the comprehensive illumination intensity at the position (x,y) at time t:

[0085]

[0086] Among them, d s is the distance between the current position of the livestock (x, y) and the sth sensor, and n is the total number of light sensors in the breeding house.

[0087] Based on the light field model, the breeding house is divided into different light areas and the light threshold L is defined. light , divide livestock activity areas according to different light intensities:

[0088] L(t,x,y)>L light Indicates areas of bright light that livestock may avoid;

[0089] L(t,x,y) <L light Indicates a shaded or shaded area, which may be a preferred activity area for livestock.

[0090] Historical location data provided by livestock collars Analyze the activity trajectories of livestock under different lighting conditions. Calculate the time livestock stay in different lighting areas during a certain period of time:

[0091]

[0092] Among them, 1(·) is an indicator function, which takes 1 when the condition is met and takes 0 otherwise. Count the time T that livestock stay in the low-light area i , and further analyze the impact of lighting conditions on their activity area selection.

[0093] Through the historical data of light and livestock individual trajectories, a multivariate regression model is used to establish a prediction model between livestock behavior and environmental conditions such as light and temperature to predict the activity area of ​​livestock in a certain period of time:

[0094]

[0095] Where P(t) is the predicted position of the livestock at time t, L(t) is the light intensity at time t, and F i (t) is other influencing factors (such as temperature, humidity, etc.), β0, β1, β i is the coefficient of the model, which indicates the contribution of each influencing factor to livestock behavior, and ∈ is the error term, which reflects the residual of the model. By normalizing historical data such as light and trajectory, the least squares optimization algorithm is used to fit the model coefficients to establish a linear relationship between livestock position and light conditions. The regression model adjusts parameters by minimizing the mean square error (MSE) to improve the prediction accuracy and realize dynamic prediction of livestock activity range in a specific time period based on real-time light and environmental data.

[0096] Furthermore, the optimized inspection paths in S5 include:

[0097] S51, using the Perceiver model to extract a unified feature representation and obtain data encoding;

[0098] S52, mapping the data encoding to the same dimension;

[0099] S53, based on the self-attention mechanism, weighted fusion of the mapped data encoding is performed to generate a comprehensive feature representation;

[0100] S54, repeat step S53 to obtain multi-level features;

[0101] S55. Optimize inspection sequence based on reinforcement learning model.

[0102] Specifically, the Perceiver model maps different types of inputs to the same latent space through a self-attention mechanism, thereby generating a comprehensive feature representation. Its key advantage is that it can efficiently process multimodal inputs and extract long-term dependent features, making it suitable for modeling and decision-making in combination with complex environmental data.

[0103] Furthermore, the model data input includes real-time image data of livestock and surrounding environment captured in the field of view of the binocular camera. t , the real-time light intensity of each breeding house monitored by the light sensor GPS modules in livestock collars provide near real-time location information of individuals Historical movement trajectories of individual livestock generated from historical location data Etc. Output different data encodings, specifically:

[0104] Image data encoding: real-time image data captured by the binocular camera I t(resolution H×W×C), including the current posture, activity and environment of the livestock, and feature extraction through convolutional neural network (CNN):

[0105] F image =CNN(I t );

[0106] in, It is the feature representation of image data;

[0107] Light intensity encoding: The real-time light intensity data of each breeding house provided by the light sensor is encoded through a layer of fully connected neural network:

[0108]

[0109] in, is the characteristic representation of light intensity data;

[0110] Location data encoding: GPS modules in livestock collars provide near-real-time location information of individuals:

[0111]

[0112] in, is a feature representation of position data;

[0113] Trajectory data encoding: The historical trajectory information of livestock extracted by the time series model LSTM is used to analyze their movement patterns:

[0114]

[0115] in, It is the feature representation of historical trajectory data;

[0116] The Perceiver model fuses input data of different modalities into a unified feature representation H(t) through a self-attention mechanism, including:

[0117] First, all input data will be mapped to the same dimension for processing in the self-attention mechanism. Suppose all modal data are combined into a vector Z(t):

[0118] Z(t)=[F image ,F light ,F position ,F trajectory ];

[0119] Each input Z(t) is mapped to a query, key, and value vector through a linear transformation. These vectors are used to calculate the weight of each element in the input data relative to other elements. Query vector Q = WQ Z(t), key vector K = W K Z(t), value vector V = W V Z(t). Where W Q , W K , W V is the weight matrix for query, key and value.

[0120] Secondly, the attention score is obtained by calculating the dot product of the query vector and the key vector and then normalizing it. The attention score represents the relevance of each element in the input data to other elements, helping the model decide which modal features should be paid more attention to. The expression is:

[0121]

[0122] Among them, d k is the dimension of the key vector, which is used to scale the dot product to prevent the gradient from disappearing due to excessive values. Softmax is used to normalize the attention score to the interval [0,1]. If the data of a certain modality (such as illumination features) is more relevant to the historical trajectory features, the attention score will be higher, which means that the features of this modality are more important.

[0123] Then, attention weighting and data fusion are performed. Through the attention score, the features of different modalities are weighted and fused to generate a new feature representation:

[0124]

[0125] Among them, α i is the attention weight obtained by soft maximization, which indicates the importance of modality i. The final output H(t) is a comprehensive feature representation of multimodal data, which contains multi-dimensional information of image features, lighting conditions and livestock trajectory information under the camera field of view.

[0126] The Perceiver model consists of n layers of self-attention modules. The feature H extracted by the nth layer (n) (t) is a high-level abstract representation of the input data at that moment. The input of each layer is the output of the previous layer. Each layer extracts features through a similar attention mechanism, but different feature patterns are paid attention to at different levels: at the initial level, the model may focus on local, lower-level features, such as local structures in a single camera frame or changes in short-term trajectories; as the number of layers increases, the model begins to extract more abstract high-level features, such as the long-term correlation between lighting and livestock behavior, the global motion trend of historical trajectories, etc.

[0127] The calculation formula for each layer is as follows:

[0128] H (l+1) (t) = Attention(H(l) (t))+H (l) (t)

[0129] Among them, H (l) (t) is the output of layer l, H (l+1) (t) is the output of the l+1th layer. Through this residual connection, the gradient vanishing problem can be avoided, ensuring that the model can effectively learn deep features.

[0130] After each layer of self-attention module, a feedforward neural network is usually added to perform nonlinear mapping on the features and further extract deep features. The calculation formula of the feedforward network is:

[0131] FFN(H(t))=ReLU(W1H(t)+b1)W2+b2

[0132] Among them, W1 and W2 are the weight matrices of the feedforward network, b1 and b2 are bias vectors, and ReLU is the activation function. FFN is used to increase the nonlinear ability of the model, which can make the extracted features richer.

[0133] Finally, the model uses an output layer PerceiverOutput to convert the feature H of the nth layer into (n) (t) is transformed into the final feature representation H final (t). Through a simple transformation layer, all the accumulated multi-level information is merged into a low-dimensional fixed-size feature vector for subsequent tasks, expressed as:

[0134] H final (t) = PerceiverOutput(H (n) (t)) = W out H (n) (t)+b out ;

[0135] Among them, W out It is used to convert H (n) (t) weight matrix for linear mapping, b out is the bias term.

[0136] The final output H final (t) It has integrated data from multiple modalities, such as camera images, light intensity, individual livestock positions, and historical trajectories, and extracted multi-level features of these data, which can be used as environmental input for reinforcement learning to guide the robot's inspection path planning in the farm.

[0137] Furthermore, the reinforcement learning model includes determining the state representation, action representation and reward function, and continuously optimizes the inspection path based on proximal policy optimization.

[0138] Specifically, the inspection robot needs to inspect all livestock individuals within a given time window (e.g., 8 hours) and observe their health status. Before the inspection begins, the initial status of all livestock individuals is "to be inspected." When the robot inspects some individuals, their status changes to "inspected," and the status of individuals that have not been observed remains "to be inspected."

[0139] The robot's inspection visit priority for each breeding house is based on the breeding house virtual inspection area V inspection The inspection order is optimized and repeated inspections are performed based on the remaining inspection time and the number of livestock individuals to be inspected.

[0140] The inspection path planning problem is modeled as a reinforcement learning problem: the inspection robot needs to make action choices based on the current environmental state (i.e., the comprehensive feature representation output by the Perceiver model) to achieve the maximum number of livestock traversals, the shortest path length, and the shortest inspection time within a given time window. The above problem can be defined as:

[0141] (1) State representation S(t)

[0142] The state space describes all possible states in the environment. The state S(t) at time t includes:

[0143] The comprehensive characteristics of the Perceiver model represent H final (t), the number of livestock individuals to be inspected in the virtual inspection area of ​​each breeding house The robot's current position P r (t), the distance D between the robot and each breeding house i (t), farm map M(t), remaining inspection time T remain (t);

[0144] The state S(t) can be expressed as:

[0145]

[0146] Among them, m is the number of breeding houses.

[0147] (2) Action representation A(t)

[0148] The action space refers to all possible operations that the robot can perform at each time t. The action A(t) at time t includes: Movement direction:

[0149] Moving distance L(t), the next farm house to be inspected is i;

[0150] The action set A(t) can be expressed as:

[0151] A(t)={D direction (t),L(t),i};

[0152] Action A(t) indicates the robot's decision on the next farm to inspect, as well as the direction and distance of the robot's movement during the inspection.

[0153] (3) Reward function R(t)

[0154] The purpose of the reward function is to comprehensively consider multiple factors in the robot inspection path planning, encourage the robot to efficiently inspect more livestock individuals, and reduce path costs and time waste. To achieve this goal, multiple reward and penalty terms can be introduced into the reward function:

[0155] ① When the robot inspects more livestock individuals, it should receive positive rewards. This reward is proportional to the number of livestock individuals to be inspected in the virtual inspection area of ​​each breeding house. Proportional to:

[0156]

[0157] Among them, r is the reward coefficient proportional to the number of inspected livestock individuals.

[0158] ② When the robot reaches a breeding house with a higher priority, it will be given an additional positive reward. The priority of the breeding house can be set manually, for example, a higher inspection priority can be given to sick animal houses and gestation houses:

[0159] R priority (t) = R·P i (t);

[0160] Among them, R is the reward coefficient proportional to the priority, P i (t) represents the priority of breeding house i.

[0161] ③ The robot should give priority to inspecting the breeding houses that are closer. Inspection of breeding houses that are farther away will bring penalties. Penalty and distance between the robot and the breeding house D i (t) is proportional to:

[0162] R cost (t) = -α·D i (t);

[0163] Among them, α is the penalty coefficient that controls the intensity of the penalty brought by the distance.

[0164] ④ Each movement of the robot has energy consumption and time cost, and the penalty should be proportional to the length L(t) of the path the robot moves;

[0165] R cost (t) = -c·L(t);

[0166] Where c is a penalty coefficient proportional to the path length.

[0167] ⑤ As the remaining inspection time decreases, the robot should be punished more to encourage it to complete the inspection task as soon as possible. Therefore, when the time decreases, the penalty should increase the sensitivity to time and control the impact of time management through the time decay function:

[0168]

[0169] Among them, β is the penalty coefficient of time management, and the penalty increases as the remaining time decreases.

[0170] The above rewards and penalties are combined to obtain the total reward function R(t) to encourage the robot to inspect livestock that have not been observed first, improve inspection efficiency and coverage; prioritize inspection of high-priority areas to ensure timely monitoring of key livestock individuals; shorten the robot's inspection path length to encourage the robot to inspect efficiently and save time and energy; as time goes by, increase penalties to encourage the robot to speed up the inspection and ensure that as many inspection tasks as possible are completed in the remaining time. The total reward function R(t) can be expressed as:

[0171]

[0172] Robot inspections need to consider multiple factors such as individual livestock locations, priorities, real-time environmental data, time management, and perform complex dynamic path planning in a continuous space. Therefore, the system needs to select a suitable deep reinforcement learning algorithm to handle dynamically changing farm inspection tasks.

[0173] Proximal Policy Optimization (PPO) is a policy gradient-based reinforcement learning algorithm that maximizes cumulative rewards by directly optimizing the policy π(A|S). In the deep reinforcement learning framework, the robot continuously interacts with the environment to learn the optimal policy π(A|S), so that the action A(t) selected in state S(t) can maximize the cumulative reward.

[0174] Optimize the policy π(A|S) to maximize the expected reward and update the policy parameters θ via gradient ascent:

[0175]

[0176] PPO uses two neural networks:

[0177] Policy Network π θ(A|S): Outputs the probability distribution of taking each action in a given state S, with parameter θ; value network π φ (A|S): Evaluate the value of a given state S with parameter φ.

[0178] The inputs of the policy network and the value network are both multimodal state representations S(t). The image input is processed through the convolutional layer, and other continuous and discrete input data are processed through the fully connected layer, and finally fused into a unified feature vector.

[0179] The core is to update the strategy π through the policy gradient θ (A∣S) and value π φ (A|S) allows strategy optimization to be carried out within a controlled range and prevents drastic changes.

[0180] The loss function of PPO consists of two parts:

[0181] Strategy loss L policy : Based on the policy gradient, update the policy π θ .

[0182]

[0183] in, is the ratio of the new strategy to the old strategy, is the advantage estimate and ∈ is a hyperparameter that controls the magnitude of the update. Represents L at time t policy The expected approximation collects sample data from the environment interaction and uses these samples to estimate the average value of the loss function to update the strategy. The clip function limits the range of changes of the new strategy relative to the old strategy to the range [1-∈, 1+∈], avoiding excessive updates of the strategy and maintaining the stability of the training process.

[0184] Value loss L value :Update the value network V through the mean square error φ .

[0185]

[0186] Among them, S t =S(t), Represents L at time t value An approximate estimate of the expected φ (S t ) is the state S estimated by the value network t value.

[0187] The final loss function is the weighted sum of policy loss and value loss:

[0188] L=L policy+c1L value -c2entropy

[0189] Among them, c1 and c2 are the weighted coefficients of loss, and entropy is the entropy term used to enhance exploration.

[0190] Sampling: The inspection robot uses the current strategy π θ (A|S) interacts with the environment and collects a series of trajectory data, including the state S of the robot at time t t , the action A selected in this state t , the reward R obtained from the environment after executing the action t , and the next state S t+1 This process generates multiple trajectories, each of which is composed of data from multiple time steps.

[0191] Calculating the advantage estimate: Using the advantage function A measure of how good an action is relative to its average expected value. Indicates that the robot is in state S t Next, execute action A t If the advantage value is positive, it means that the action performs better than expected, otherwise it is worse.

[0192]

[0193] Among them, γ is a discount factor used to weigh long-term rewards and short-term rewards.

[0194] Update the policy and value networks: Through back-propagation, the loss function is used to optimize the policy network and value network so that they can better fit the advantage estimate and state value.

[0195] Iterative optimization: Continuously sample, evaluate, and optimize strategies until they converge. Continuously optimize the robot's inspection path so that it can complete all tasks within a given time window.

[0196] Furthermore, the health status analysis in S6 includes:

[0197] Based on normal Establish a support vector machine (SVM) baseline model. The deviation from the baseline model is used to evaluate health status. Health indicators are defined based on domain knowledge, such as activity frequency, residence time, etc. The robot uses infrared cameras to obtain individual body temperature data during the inspection process and conducts comprehensive analysis. It combines the analysis of individual livestock behavior patterns, activity frequency, residence time and other information to identify abnormal behaviors related to the disease, such as lameness, loss of appetite, overactivity, and long-term immobility, assess the individual's health status, and achieve early disease warning and individual status assessment.

[0198] When the system detects an abnormality (such as abnormal body temperature or abnormal behavior or activity), it will record the identification number of the abnormal individual, the detection time, the breeding house to which it belongs, and the type of abnormal status, and upload the relevant information to the farm management system via the wireless communication module in real time, triggering a real-time alarm to remind managers to take timely measures to ensure the health of individual livestock.

[0199] Furthermore, as shown in Tables 1 and 2, the inspection robot establishes a task list to be inspected for each breeding house, recording the number of livestock individuals to be inspected in the house and the detailed health records of each livestock individual. Each breeding house corresponds to a task, and the inspection status of all breeding houses is set to "unfinished" in the initial state. After completing the inspection task of each breeding house, the robot updates the observed livestock individual files and the remaining number of individuals to be inspected in the breeding house in real time. When all livestock individuals in a breeding house are inspected, the inspection status of the house is updated to "completed", and there is no need to inspect the house again in the future. If there are still livestock individuals that have not been inspected in a breeding house, that is, the actual number of observations is less than the total number registered, the robot marks the breeding house as "partially completed". Subsequently, the robot continues to take the "partially completed" and "unfinished" breeding houses as inspection targets according to the inspection task list and repeats the inspection.

[0200] Table 1 List of breeding houses to be inspected

[0201] Breeding house number Total number of livestock in the house Number of livestock to be inspected in the virtual inspection area Inspection status 001 20 5 Completed 002 40 15 Partially Completed 003 30 10 Unfinished

[0202] Table 2 Livestock individual health records

[0203]

[0204] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An autonomous inspection system for farm inspection robots based on reinforcement learning, characterized in that: Includes signal-connected inspection robots, positioning collars and cloud servers.

2. A method for autonomous inspection of farm inspection robots based on reinforcement learning, characterized in that: The following steps are involved: S1. The inspection robot constructs multiple farm environment maps, divides virtual inspection areas and obtains environmental data; S2, the inspection robot receives the real-time location information of the livestock positioning collar, obtains the livestock's historical location data, establishes a trajectory analysis model, and obtains livestock trajectory analysis data; S3, establishing a trajectory prediction model based on livestock trajectory analysis data and environmental data to predict livestock activity areas in a specific time period; S4. Determine the area to be inspected based on the livestock activity area and real-time location information; S5. The inspection robot determines to build a learning reinforcement model to optimize the inspection path according to the number of livestock, distance and time in the inspection area, and detects the physical condition of the livestock; S6. Determine the health status of the livestock based on the livestock physical condition and livestock trajectory analysis data, and upload the information to the cloud server.

3. The method for autonomous inspection of a farm inspection robot based on reinforcement learning according to claim 2 is characterized in that: The livestock trajectory analysis data in S2 include: S21. Use LSTM to learn and extract the historical trajectory of livestock from the time series position data to obtain the trajectory characteristics of individual livestock in the time series; S22. Analyze the distance and speed between different time points to obtain the movement characteristics of livestock.

4. The method for autonomous inspection of a farm inspection robot based on reinforcement learning according to claim 2 is characterized in that: Livestock trajectory prediction in S3 includes: S31, deploying multiple light sensors in different areas of the breeding house to obtain light data corresponding to the sensors; S32, constructing a lighting distribution model for the entire breeding house according to the lighting data, and dividing different lighting areas based on the model data; S33. Based on the historical data of light and individual livestock trajectories, a multivariate regression model is used to establish a prediction model between livestock behavior and light, and dynamically predict the activity range of livestock in a specific time period.

5. The method for autonomous inspection of a farm inspection robot based on reinforcement learning according to claim 2 is characterized in that: The optimized inspection paths in S5 include: S51, using the Perceiver model to extract a unified feature representation and obtain data encoding; S52, mapping the data encoding to the same dimension; S53, based on the self-attention mechanism, weighted fusion of the mapped data encoding is performed to generate a comprehensive feature representation; S54, repeat step S53 to obtain multi-level features; S55. Optimize inspection sequence based on reinforcement learning model.

6. The method for autonomous inspection of a farm inspection robot based on reinforcement learning according to claim 5 is characterized in that: The reinforcement learning model includes determining state representation, action representation and reward function, and continuously optimizes the inspection path based on proximal strategy optimization.

Images