Convolutional neural network based adaptive navigation path planning system for field robots

Abstract

The application discloses a field robot adaptive navigation path planning system based on a convolutional neural network and belongs to the technical field of agricultural robots.The application solves the problem that the prior art cannot meet the demand of efficient and accurate operation of the field robot, identifies and marks key entities in the field through the convolutional neural network, perceives environmental changes in advance, enables the robot to react in advance, avoids path planning failure or operation interruption, dynamically adjusts path planning weights according to a task type and a real-time environment, flexibly adapts to different demands and conditions, guarantees operation continuity and stability, and defines a balance mechanism of automatic decision and manual intervention, combines a three-level abnormality disposal strategy, and ensures quick and effective response under different abnormal conditions, which not only improves the intelligent level of the system, but also significantly enhances the operation efficiency and adaptability of the robot in a complex environment, and effectively promotes the development of agricultural automation and intelligence.

Description

Technical Field

[0001] This invention relates to the field of agricultural robot technology, specifically to an adaptive navigation path planning system for field robots based on convolutional neural networks. Background Technology

[0002] With the advancement of agricultural modernization, field robots are being used more and more widely in agricultural production.

[0003] However, the field environment is complex and varied, with many factors such as obstacles, terrain undulations, and differences in crop growth, making it difficult for traditional navigation path planning methods to meet the needs of efficient and precise operation of field robots.

[0004] Therefore, to meet existing needs, an adaptive navigation path planning system for field robots based on convolutional neural networks is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive navigation path planning system for field robots based on convolutional neural networks. This system identifies and marks key entities in the field using convolutional neural networks, enabling the robot to anticipate environmental changes and react in advance, thus avoiding path planning failures or operational interruptions. It dynamically adjusts path planning weights according to task type and real-time environment, flexibly adapting to different needs and conditions to ensure operational continuity and stability. Furthermore, it defines a balance mechanism between automated decision-making and human intervention, combined with a three-level anomaly handling strategy, ensuring rapid and effective response under different abnormal conditions. This not only improves the system's intelligence level but also significantly enhances the robot's operational efficiency and adaptability in complex environments, powerfully promoting the development of agricultural automation and intelligence, and solving the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An adaptive navigation path planning system for field robots based on convolutional neural networks includes:

[0008] The twin modeling unit is configured to construct a digital twin model that reflects the dynamic changes of the field environment in real time based on multi-dimensional data of the field environment;

[0009] The environmental perception unit is configured to construct a convolutional neural network for environmental perception based on historical data. The image data of the field environment is used as input, and the features in the image are automatically extracted based on the convolutional neural network. The output is the recognition results of obstacles, crop areas and passable areas in the environment.

[0010] The adaptive path planning unit is configured to perform path planning based on environmental information output by the digital twin model and recognition results output by the convolutional neural network, using a fuzzy logic algorithm combined with the kinematic and dynamic constraints of the field robot and path factors.

[0011] Furthermore, the adaptive path planning unit includes:

[0012] The thread creation module is configured to use a convolutional neural network to identify and label key entities in the field environment, assigning a unique identifier to each entity;

[0013] In the digital twin model, a digital thread link is created for each key entity to record and update the status information of each key entity in real time, including but not limited to position, shape, speed and direction, and synchronize it to the corresponding digital thread link;

[0014] Acquire historical state data for each key entity, including but not limited to changes in position, shape, and velocity, and analyze its motion patterns and trends.

[0015] A predictive model is constructed using long short-term memory networks to predict possible states in the near future, including: using wind speed and direction data combined with the physical characteristics of crops to predict the swaying amplitude and direction of crops; and analyzing the movement trajectory and speed of suspected animals in the area to predict their movement probability and direction.

[0016] The prediction results will be output in the form of structured data, including but not limited to the location and morphological change range of each key entity at future time points, providing environmental auxiliary data for path planning.

[0017] Furthermore, the adaptive path planning unit further includes:

[0018] The path adjustment module is configured to obtain the current task type of the field robot and evaluate the priority of the task based on its importance and urgency.

[0019] Based on current environmental information and task requirements, the target weights of the path planning are dynamically adjusted to meet diverse field operation needs.

[0020] During the route planning process, changes in the field environment are monitored in real time, and environmental information is quickly updated through a digital twin model;

[0021] Based on the updated environmental information, the task requirements and path planning objective weights are reassessed, and the path planning strategy is dynamically adjusted.

[0022] Furthermore, the adaptive path planning unit further includes:

[0023] The intelligent decision-making module is configured to define a balance mechanism between automated decision-making and human intervention, and to divide tasks into three priorities: high, medium, and low, based on their complexity, risk, and importance.

[0024] When making automated decisions, high-priority tasks should be prioritized for human intervention, while low-priority tasks should be prioritized for automated decision-making.

[0025] Create a three-tiered anomaly handling strategy, adopting different handling methods based on the severity and complexity of the anomaly.

[0026] Furthermore, based on current environmental information and task requirements, the target weights of the path planning are dynamically adjusted, including:

[0027] Based on the task requirements, define multiple target weights for path planning, including: path shortestness weight, smoothness weight, and safety weight.

[0028] Real-time acquisition of environmental information from the digital twin model to assess the impact of the current environment on path planning objectives;

[0029] Define fuzzy logic rules and convert environmental information into fuzzy sets;

[0030] Based on fuzzy logic rules, the rules corresponding to the current environmental information and task requirements are matched, and reasoning calculations are performed to obtain fuzzy reasoning results.

[0031] The fuzzy inference results are converted into adjustment values ​​for the target weights, and the target weights are dynamically adjusted.

[0032] Furthermore, the adaptive path planning unit further includes:

[0033] The lifecycle management module is configured to automatically create a digital thread link when a new critical entity is detected.

[0034] For digital thread links that have not been updated for a long time, have left the area of ​​interest, or whose entity status has stabilized, delete them from the archived database to free up computing resources.

[0035] Furthermore, the environmental sensing unit includes:

[0036] The multi-scale feature extraction module is configured to extract both local and global features from an image simultaneously, and introduces an attention mechanism to enable the convolutional neural network to automatically focus on key regions in the image.

[0037] Furthermore, it also includes:

[0038] The data acquisition unit is configured to install multiple sensors on the field robot, including: lidar for accurately measuring obstacle distances and terrain heights; ultrasonic sensors for detecting obstacles at close range; and soil moisture sensors for monitoring soil moisture distribution.

[0039] The data processing unit is configured to use multimodal fusion technology to fuse image data, lidar data, ultrasonic data and soil moisture data to generate multimodal fused data.

[0040] Furthermore, predictive models are constructed using long short-term memory networks, including:

[0041] Obtain the model training dataset; the model training dataset includes LiDAR data, ultrasonic data, soil moisture data, and image data;

[0042] Feature extraction is performed on the training dataset, and the extracted features are fused to obtain multimodal fusion features;

[0043] Construct a hierarchical LSTM architecture; the bottom layer is a 2-layer bidirectional LSTM with temporal residual connections, which takes the final fused features as input to capture short-term fine-grained temporal dependencies and outputs the bottom-layer features.

[0044] The middle layer performs adaptive time-frequency decomposition on the low-level features, constructs time-frequency attention weights to dynamically fuse time-domain and frequency-domain features, and outputs the fused features of the middle layer.

[0045] The upper layer extracts shared features through a fully connected layer and constructs a dual prediction branch;

[0046] Among them, the crop swing prediction branch inputs shared features and prior knowledge of crop physical characteristics, and outputs swing amplitude and horizontal direction angle; the animal movement prediction branch inputs shared features and suspected animal area features, and outputs movement probability, horizontal movement direction, and movement speed.

[0047] Construct a multi-dimensional loss function; the multi-dimensional loss function includes multi-task weighted loss and modality consistency loss; iteratively train the model based on the multi-dimensional loss function until the training result meets the requirements, and obtain the prediction model.

[0048] Furthermore, the data processing unit includes:

[0049] Acquisition sub-units are used to acquire multi-source heterogeneous raw data;

[0050] The preprocessing subunit is used for:

[0051] The various types of raw data from the multi-source heterogeneous raw data are uniformly converted into a matrix format of timestamp-feature dimension, and all data are aligned in time dimension based on timestamps; the non-stationary index corresponding to each modality is calculated, and non-stationary calibration is performed based on the calibration formula of modality affine parameters, and the calibrated data is output; modal multi-scale windows are configured based on the sampling rate differences of each modality, and the multi-scale features of all modalities are unified into 64 time step windows through upsampling linear interpolation and downsampling mean pooling operations to obtain multi-scale unified features;

[0052] The first fusion subunit is used for:

[0053] Adaptive time-frequency decomposition is performed on the highly periodic modes in the calibrated data. The 64-point fast Fourier transform is used for the highly periodic data and the 32-point fast Fourier transform is used for the weakly periodic data. The amplitude and phase features of the main frequency are extracted and concatenated with the time domain features to form dual-domain features. The dual-domain features are then fused with the multi-scale unified features to obtain a standardized multi-scale feature sequence.

[0054] The enhancement subunit is used to perform standardized multi-scale feature processing on each mode through an intra-modal feature enhancement branch composed of an improved MambaFFN module. The improved MambaFFN module includes a linear feature enhancement branch and a nonlinear feature enhancement branch. The outputs of the two branches are dynamically weighted and fused according to the modal feature signal-to-noise ratio to obtain the intra-modal enhanced features.

[0055] The second fusion subunit is used to construct a cross-modal association prior matrix. The matrix elements are the preset association strengths between each modality. The intra-modal enhancement features of each modality are weighted and summed according to the association strength to obtain the cross-modal association features.

[0056] The third fusion subunit is used to concatenate the intra-modal enhancement features and cross-modal correlation features of each modality, input them into the modal fully connected layer, unify the feature dimension to 1024 dimensions, and output the final feature sequence of each modality;

[0057] The filtering subunit is used to: calculate the credibility of each modal feature data, filter each modal feature based on the credibility, and obtain the filtered features;

[0058] The fourth fusion subunit is used to align all filtered features to a uniform length using a dynamic time warping algorithm, construct a cross-modal attention matrix, calculate the correlation strength between any two modal features at each time step, and fuse all modal features by weighting the correlation strength to obtain multimodal fused features.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] 1. In this invention, key entities in the field environment are identified and labeled by a convolutional neural network, which can detect environmental changes in advance and enable the robot to react in advance, avoiding path planning failure or operation interruption caused by sudden environmental changes; and the target weight of path planning is dynamically adjusted according to the task type and real-time environmental information to ensure that the path planning strategy can flexibly adapt to different task requirements and environmental conditions, thereby improving the continuity and stability of the operation.

[0061] 2. In this invention, by defining a balance mechanism between automated decision-making and human intervention, it ensures that human intervention can be reasonably introduced during critical decisions, while fully leveraging the advantages of automation in routine tasks. Furthermore, by employing a three-level anomaly handling strategy, the anomaly handling methods are further refined to ensure that the system can react quickly and effectively to anomalies of varying severity. This not only improves the system's intelligence level but also significantly enhances the efficiency and adaptability of field robots in complex environments, thus promoting the development of agricultural automation and intelligence. Attached Figure Description

[0062] Figure 1 This is a flowchart of the adaptive navigation path planning system for field robots based on convolutional neural networks according to the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] To address the technical challenge of existing technologies where the complex and varied field environment, including obstacles, terrain undulations, and differences in crop growth, renders traditional navigation path planning methods inadequate for meeting the demands of efficient and precise field robot operations, please refer to [link to relevant documentation]. Figure 1 This embodiment provides the following technical solution:

[0065] An adaptive navigation path planning system for field robots based on convolutional neural networks includes:

[0066] The data acquisition unit is configured to install multiple sensors on the field robot, including: lidar for accurately measuring obstacle distance and terrain height; ultrasonic sensors for detecting obstacles at close range; and soil moisture sensors for monitoring soil moisture distribution.

[0067] The data processing unit is configured to employ multimodal fusion technology, such as an attention mechanism fusion network, to automatically learn the importance and correlation of data from different sensors. It fuses image data, LiDAR data, ultrasonic data, and soil moisture data to generate richer and more accurate multimodal data. Through multimodal data fusion, the field environment can be perceived more comprehensively, not only identifying obstacles and crop areas but also sensing key information such as soil moisture. This provides a more comprehensive basis for path planning and improves the robustness and accuracy of the perception results.

[0068] The twin modeling unit is configured to construct a digital twin model that reflects the dynamic changes of the field environment in real time based on multi-dimensional data of the field environment. The multi-dimensional data includes information such as terrain, crop distribution, and obstacle location. The digital twin model supports multi-scale visualization, from macro field layout to micro crop growth details, and maintains a high degree of consistency with the physical field environment, providing comprehensive environmental information support for path planning.

[0069] The environmental perception unit is configured to construct a convolutional neural network for environmental perception based on historical data. Image data of the field environment is used as input. The convolutional neural network automatically extracts features from the images, such as the shape of obstacles (rocks, animals), crop texture, segmentation of exposed soil areas, and preliminary classification of crop growth. It outputs the identification results of obstacles, crop areas, and passable areas in the environment, achieving rapid and accurate perception of complex field environments and ensuring adaptability to field scenarios under different seasons and lighting conditions. The environmental perception unit includes:

[0070] The multi-scale feature extraction module is configured to simultaneously extract local features in the image, such as obstacle shapes, and global features, such as crop distribution. It also introduces an attention mechanism to enable the convolutional neural network to automatically focus on key regions in the image, thereby improving the accuracy and efficiency of perception.

[0071] The adaptive path planning unit is configured to perform path planning based on environmental information output by the digital twin model and recognition results output by the convolutional neural network. It employs a fuzzy logic algorithm combined with the kinematic and dynamic constraints of the field robot, as well as path factors, including path shortestness, smoothness, and safety. The adaptive path planning unit includes:

[0072] The thread creation module is configured to use convolutional neural networks to identify and label key entities in the field environment, assigning a unique identifier to each entity. Within the digital twin model, it creates digital thread links for each key entity, such as field robots, major obstacles, and crops. This enables the system to manage multiple dynamic elements in the field environment with finer detail, providing more comprehensive and accurate environmental information support for path planning. It records and updates the state information of each key entity in real time, including but not limited to position, shape, speed, and direction, and synchronizes it to the corresponding digital thread link. It acquires historical state data for each key entity, including but not limited to changes in position, shape, and speed, analyzing its motion patterns and trends. Finally, it uses a long short-term memory network to build a predictive model to predict its short-term future. For example, the possible states in the next 5-10 seconds include: using wind speed and direction data combined with the physical characteristics of crops to predict the swaying amplitude and direction of crops; analyzing the movement trajectory and speed of suspected animal areas to predict their movement probability and direction; outputting the prediction results in the form of structured data, including but not limited to the location and morphological change range information of each key entity at future time points, providing environmental auxiliary data for path planning; for example, the output result is: the crop swaying area is expanding 3 meters ahead, and it is recommended to slow down; the suspected shadow area on the right becomes a passable area within 10 seconds; through thread creation and state prediction, the system can perceive the dynamic changes of the field environment in advance, providing forward-looking auxiliary data for path planning, enabling field robots to react in advance and avoid path planning failure or operation interruption due to sudden situations.

[0073] The lifecycle management module is configured to automatically create digital thread links when a new critical entity, such as a newly appearing obstacle, is detected; and to delete digital thread links that have not been updated for a long time, have left the area of ​​interest (e.g., the robot has passed by), or whose entity status has stabilized (e.g., a marked stone) from the archived database, thus freeing up computing resources.

[0074] The path adjustment module is configured to: acquire the current task type of the field robot (e.g., fertilization tasks may prioritize path coverage, while harvesting tasks prioritize efficiency and safety); assess task priority based on importance and urgency (e.g., urgent pest and disease control tasks may have higher priority and require prioritized path planning and execution); dynamically adjust the target weights of path planning based on current environmental information and task requirements (e.g., balancing path shortest distance, smoothness, and safety) to improve task execution efficiency and quality, meeting diverse field operation needs; monitor changes in the field environment in real time during path planning (e.g., newly appearing obstacles or crop lodging), and quickly update environmental information through a digital twin model to ensure high consistency between the model and the physical field environment; reassess task requirements and path planning target weights based on updated environmental information, and dynamically adjust the path planning strategy; replan the path to avoid new obstacles; and adjust the operation sequence or path when crop lodging affects operations to ensure the robot always travels along the optimal path, guaranteeing efficient task completion. Its steps include:

[0075] Based on task requirements, multiple objective weights are defined for path planning, including: path shortest distance weight, smoothness weight, and safety weight. For example, if the current task is sowing, and the path planning prioritizes coverage and uniformity, the objective weights are allocated as: path shortest distance weight (0.3), smoothness weight (0.4), and safety weight (0.3). If the current task is harvesting, and the path planning prioritizes efficiency and safety, the objective weights are allocated as: path shortest distance weight (0.5), smoothness weight (0.3), and safety weight (0.2). Environmental information from the digital twin model, including obstacle locations, crop distribution, and terrain undulations, is acquired in real time for evaluation. The impact of the current environment on path planning objectives provides a basis for dynamically adjusting objective weights. For example, higher obstacle density has a greater impact on safety; tall crops may affect path smoothness, and fallen crops may increase path safety risks; in areas with significant terrain undulations, path smoothness requirements are higher, while the robot's climbing ability and stability also need to be considered. Fuzzy logic rules are defined to convert environmental information into fuzzy sets, such as dividing obstacle density into low, medium, and high fuzzy sets. Fuzzy logic rules include rules for adjusting safety weights: if obstacle density is high, such as more than 3 obstacles per square meter, the safety weight is increased. Increase by 0.2; If obstacle density is low, such as less than one obstacle per square meter, decrease the safety weight, such as by 0.1; If the lodged crop area is large, such as exceeding 50% of the path width, increase the safety weight, such as by 0.1; If the terrain is undulating, such as a slope exceeding 15 degrees, increase the safety weight, such as by 0.1; For path shortest distance weight adjustment rules: If obstacle density is low and terrain is flat, such as a slope less than 5 degrees, increase the path shortest distance weight, such as by 0.2; If obstacle density is high or terrain is undulating, decrease the path shortest distance weight, such as by 0.1; For smoothness weight adjustment rules: ... If the terrain is undulating, such as a slope exceeding 10 degrees, the smoothness weight is increased by 0.2. If crops are unevenly distributed and have large height differences, such as a maximum height difference exceeding 1 meter, the smoothness weight is increased by 0.1. Based on fuzzy logic rules, the system matches the rules corresponding to the current environmental information and task requirements, performs inference calculations, and obtains fuzzy inference results. The fuzzy inference results are converted into adjustment values ​​for the target weights, and the target weights are dynamically adjusted. By dynamically adjusting the target weights, the system can flexibly adjust the path planning strategy according to the real-time environment and task requirements, ensuring that the optimal path can be generated in different scenarios, thus improving the adaptability and flexibility of path planning.

[0076] The beneficial effects achieved by the above are as follows: by identifying and labeling key entities in the field environment through convolutional neural networks, the robot can perceive environmental changes in advance, enabling it to react in advance and avoid path planning failures or work interruptions caused by sudden environmental changes; and by dynamically adjusting the target weights of path planning according to task type and real-time environmental information, the path planning strategy can flexibly adapt to different task requirements and environmental conditions, thereby improving the continuity and stability of the operation.

[0077] The intelligent decision-making module is configured to define a balance mechanism between automated decision-making and human intervention. Based on the complexity, risk, and importance of the task, tasks are categorized into high, medium, and low priorities. High-priority tasks, such as emergency pest and disease control, prioritize human intervention during automated decision-making; low-priority tasks, such as routine data collection, prioritize automated decision-making. This ensures that human intervention can be reasonably introduced in critical decisions while fully leveraging the advantages of automation in routine tasks. A three-tiered anomaly handling strategy is created, employing different processing methods based on the severity and complexity of the anomaly. For example, Level L1: For predictable and pre-planned common anomalies, such as small-scale temporary obstacles or minor sensor malfunctions, these are classified as Level L1 and pre-defined strategies are directly invoked. The system processes and records the process and results to reduce the frequency of human intervention and improve operational efficiency. Level 2: For complex, low-determinism anomalies, execution is paused, and multiple handling suggestions are sent to the remote monitor, along with a utility analysis of each option. For example, for large areas of unknown disease, suggestions include: A. Detour (time +15%), B. Slow-speed traverse and sample collection, C. Request on-site confirmation, to be selected by personnel. Level 3: For extreme situations, such as when the robot is predicted to overturn, an emergency stop is immediately initiated, and control is completely transferred to human intervention. Through intelligent decision-making, it ensures that human intervention can be reasonably introduced in critical decisions, while fully leveraging the advantages of automation in routine tasks to ensure that the system can react quickly and effectively to anomalies.

[0078] The beneficial effects achieved by the above are as follows: By defining a balance mechanism between automated decision-making and human intervention, it ensures that human intervention can be reasonably introduced in critical decisions, while giving full play to the advantages of automation in routine tasks; and by using a three-level anomaly handling strategy to further refine the anomaly handling methods, it ensures that the system can respond quickly and effectively to anomalies of different severity levels. This not only improves the intelligence level of the system, but also significantly enhances the operating efficiency and adaptability of field robots in complex environments, thus promoting the development of agricultural automation and intelligence.

[0079] Working principle: The robot reflects the dynamic changes of the field environment in real time through a digital twin model, extracts image features using a convolutional neural network, and identifies obstacles, crop areas, and passable areas; it uses a fuzzy logic algorithm for path planning, creates digital thread links for key entities, and updates status information in real time; it dynamically adjusts path weights based on task priority and environmental information, balances automated decision-making with human intervention according to task risk and complexity, and adopts a three-level anomaly handling strategy, which significantly improves the efficiency and adaptability of the field robot in complex environments.

[0080] Building predictive models using long short-term memory networks includes:

[0081] Obtain the model training dataset; the model training dataset includes LiDAR data, ultrasonic data, soil moisture data, and image data;

[0082] Feature extraction is performed on the training dataset, and the extracted features are fused to obtain multimodal fusion features;

[0083] Construct a hierarchical LSTM architecture; the bottom layer is a 2-layer bidirectional LSTM with temporal residual connections, which takes the final fused features as input to capture short-term fine-grained temporal dependencies and outputs the bottom-layer features.

[0084] The middle layer performs adaptive time-frequency decomposition on the low-level features, constructs time-frequency attention weights to dynamically fuse time-domain and frequency-domain features, and outputs the fused features of the middle layer.

[0085] The upper layer extracts shared features through a fully connected layer and constructs a dual prediction branch;

[0086] Among them, the crop swing prediction branch inputs shared features and prior knowledge of crop physical characteristics, and outputs swing amplitude and horizontal direction angle; the animal movement prediction branch inputs shared features and suspected animal area features, and outputs movement probability, horizontal movement direction, and movement speed.

[0087] Construct a multi-dimensional loss function; the multi-dimensional loss function includes multi-task weighted loss and modality consistency loss; iteratively train the model based on the multi-dimensional loss function until the training result meets the requirements, and obtain the prediction model.

[0088] In this embodiment, the bottom layer is used for feature capture, the middle layer is used for time-frequency fusion, and the top layer is used for multi-task prediction, forming a three-layered architecture.

[0089] In this embodiment, the bottom layer is a short-term temporal feature capture LSTM layer, which uses two bidirectional LSTM layers with a hidden layer dimension of 1024 and a dropout rate of 0.2 to fuse multimodal features. Captures short-term, fine-grained temporal dependencies and outputs underlying features. Simultaneously, a temporal residual connection is added, and the input features are element-wise added to the LSTM output features: ; The feature vector output by the underlying LSTM; It is a 2-layer bidirectional LSTM network; This serves as the final feature input for multimodal fusion.

[0090] In this embodiment, the temporal features are adopted from the bottom layer output. Preserve the dynamic changes in time sequence.

[0091] In this embodiment, the frequency domain features are extracted by adaptive time-frequency decomposition of the underlying features, and the dimension is compressed to 1024 by global average pooling.

[0092] In this embodiment, the time-frequency attention weight is used to balance the contributions of time-domain and frequency-domain features: ; For time-frequency attention weights; This is the temporal feature vector output by the underlying LSTM; It is a frequency domain eigenvector; ;in, For frequency domain characteristics, For the Sigmoid activation function, A linear layer that outputs scalar features; a fused feature layer that outputs mid-level features. The autocorrelation coefficient of the sequence is calculated to determine the strength of the time dependence.

[0093] In this embodiment, the upper layer includes a multi-task prediction head, which outputs shared features through a fully connected layer, ReLU activation, and batch normalization. .

[0094] In this embodiment, the dual prediction branches include a crop swing prediction branch and an animal movement prediction branch.

[0095] In this embodiment, the crop swing prediction branch will share features. The input features are spliced ​​together with the prior physical characteristics of crops and then passed through two fully connected layers to output the swing amplitude and horizontal angle of the crops.

[0096] In this embodiment, the animal movement prediction branch will share features. The features of the suspected animal region are concatenated and used as input features. The output consists of two fully connected layers, which output the probability of movement, the direction of horizontal movement, and the speed of movement.

[0097] In this embodiment, a learnable interaction matrix is ​​added between the fully connected layers of the crop swing prediction branch and the animal movement prediction branch to constrain the hidden layer features of the two branches. ,in, This is the balance coefficient; A learnable interaction matrix; The hidden layer features of the crop swinging branch after interaction; Features of the hidden layer of the original crop swing branch; Hidden layer features for predicting animal movement branches.

[0098] In this embodiment, the multi-task weighted loss function, ; For multi-task weighted loss function; Losses due to crop swaying; Predict losses for animal movement.

[0099] In this embodiment, crop sway loss is calculated using MSE loss, with MSE representing the sway amplitude and angular period loss representing the horizontal angle. ;

[0100] in, Losses due to crop swaying; To predict the horizontal direction angle; This is the true horizontal direction angle; The number of time steps; The predicted swing amplitude; This represents the actual swing amplitude.

[0101] In this embodiment, the animal movement loss is calculated as follows: movement probability is represented by binary cross-entropy, movement direction by angular periodicity loss, and movement speed by MSE.

[0102] ; Predict total losses for animal movement; For binary cross-entropy loss; To predict the probability of movement; This represents the actual movement probability; This refers to the periodic loss of the angle in the horizontal movement direction. To predict the horizontal movement direction angle; This represents the actual horizontal movement direction angle; To predict movement speed; This represents the actual movement speed.

[0103] In this embodiment, modal consistency loss is calculated by using the Pearson correlation coefficient of the mean vectors of each modal feature.

[0104] ;in, The number of modes; (⋅) represents the mean of the feature dimension; For the preset correlation matrix; This represents the modal consistency loss. (⋅) represents the Pearson correlation coefficient; This represents the total number of time steps.

[0105] In this embodiment, the multi-dimensional loss function is: ; For multi-dimensional loss; Weighted loss for multiple tasks; This represents the modal consistency loss.

[0106] The working principle and beneficial effects of the above technical solution are as follows: The model integrates multimodal information such as lidar data, ultrasonic data, soil moisture data, and image data, enabling it to comprehensively perceive the environment from different angles; compared with models based on a single data source, it can capture richer features, greatly improving the model's understanding and analysis capabilities of complex scenes, providing a solid data foundation for accurately predicting crop swaying and animal movement; feature extraction and fusion of the training dataset yields multimodal fusion features, reducing data redundancy while enhancing feature expressiveness, allowing the model to better mine potential correlations between different data, improving prediction accuracy and reliability; a two-layer bidirectional LSTM with temporal residual connections can effectively capture short-term fine-grained temporal dependencies, and residual connections help alleviate the gradient vanishing problem, making the model more stable when processing long-sequence data, enabling it to more accurately learn short-term change patterns in the data, providing high-quality underlying features for subsequent predictions; through adaptive time-frequency decomposition and time-frequency attention weights, Dynamically fusing time-domain and frequency-domain features fully utilizes the information in both domains, capturing both periodic and transient characteristics, enhancing the model's ability to analyze complex signals, and improving feature representation. Constructing dual prediction branches to predict crop swaying and animal movement separately combines shared features with their respective prior information, enabling refined modeling for different prediction tasks. This makes the model more targeted and professional, improving prediction accuracy and effectiveness. Multi-dimensional loss functions include multi-task weighted loss and modality consistency loss. Multi-task weighted loss balances the importance of different prediction tasks, allowing the model to achieve good performance across multiple tasks. Modality consistency loss ensures the consistency of different modalities in the feature space, improving the model's ability to fuse and generalize multi-modal data. Through iterative training based on multi-dimensional loss functions, the model continuously optimizes its parameters until it achieves satisfactory training results, improving training efficiency and prediction performance.

[0107] The data processing unit includes:

[0108] Acquisition sub-units are used to acquire multi-source heterogeneous raw data;

[0109] The preprocessing subunit is used for:

[0110] The various types of raw data from the multi-source heterogeneous raw data are uniformly converted into a matrix format of timestamp-feature dimension, and all data are aligned in time dimension based on timestamps; the non-stationary index corresponding to each modality is calculated, and non-stationary calibration is performed based on the calibration formula of modality affine parameters, and the calibrated data is output; modal multi-scale windows are configured based on the sampling rate differences of each modality, and the multi-scale features of all modalities are unified into 64 time step windows through upsampling linear interpolation and downsampling mean pooling operations to obtain multi-scale unified features;

[0111] The first fusion subunit is used for:

[0112] Adaptive time-frequency decomposition is performed on the highly periodic modes in the calibrated data. The 64-point fast Fourier transform is used for the highly periodic data and the 32-point fast Fourier transform is used for the weakly periodic data. The amplitude and phase features of the main frequency are extracted and concatenated with the time domain features to form dual-domain features. The dual-domain features are then fused with the multi-scale unified features to obtain a standardized multi-scale feature sequence.

[0113] The enhancement subunit is used to perform standardized multi-scale feature processing on each mode through an intra-modal feature enhancement branch composed of an improved MambaFFN module. The improved MambaFFN module includes a linear feature enhancement branch and a nonlinear feature enhancement branch. The outputs of the two branches are dynamically weighted and fused according to the modal feature signal-to-noise ratio to obtain the intra-modal enhanced features.

[0114] The second fusion subunit is used to construct a cross-modal association prior matrix. The matrix elements are the preset association strengths between each modality. The intra-modal enhancement features of each modality are weighted and summed with the intra-modal enhancement features of other modalities according to the association strength to obtain the cross-modal association features.

[0115] The third fusion subunit is used to concatenate the intra-modal enhancement features and cross-modal correlation features of each modality, input them into the modal fully connected layer, unify the feature dimension to 1024 dimensions, and output the final feature sequence of each modality;

[0116] The filtering subunit is used to: calculate the credibility of each modal feature data, filter each modal feature based on the credibility, and obtain the filtered features;

[0117] The fourth fusion subunit is used to align all filtered features to a uniform length using a dynamic time warping algorithm, construct a cross-modal attention matrix, calculate the correlation strength between any two modal features at each time step, and fuse all modal features by weighting the correlation strength to obtain multimodal fused features.

[0118] In this embodiment, the non-stationary index refers to different non-stationary indices corresponding to different data. For example, for lidar data, it is variance plus trend term slope; the trend term slope is calculated using linear regression with a 10-second sliding window; for soil moisture data, it is variance plus periodicity index; the periodicity index is calculated using the main frequency amplitude of FFT with a 60-second window; for ultrasonic data, it is variance plus instantaneous fluctuation amplitude; the instantaneous fluctuation amplitude is the standard deviation of a 50ms window; and for image target feature data, it is variance plus target area change rate; the target area change rate is the relative change between adjacent frames.

[0119] In this embodiment, the lidar data calibration formula is as follows: ;in, The average value over a 10-second sliding window; The standard deviation is for a 10-second sliding window. The trend slope; This refers to the current time point; , For learnable affine parameters; This is the calibrated lidar data; This is the raw lidar data.

[0120] In this embodiment, the soil moisture data calibration formula is as follows: ;in, Main frequency, , These are the periodic amplitude and phase, respectively; , For learnable affine parameters; For calibrated soil moisture data; This is the original soil moisture data; The mean of the sliding window; This represents the standard deviation of the sliding window.

[0121] In this embodiment, the ultrasonic data calibration formula is as follows: ;in, This refers to the instantaneous fluctuation amplitude; This is a threshold value used to determine whether instantaneous fluctuations need to be compensated. , For learnable affine parameters; The calibrated ultrasonic data; This is the raw ultrasound data; The mean of the sliding window; This represents the standard deviation of the sliding window.

[0122] In this embodiment, the image data calibration formula is: ;in, The target area change rate; , For learnable affine parameters; For calibrated image feature data; This refers to the original image feature data; The mean of the sliding window; The standard deviation of the sliding window; The target area change rate, For time, it is used to compensate for changes in area over time.

[0123] In this embodiment, ultrasonic and lidar are high-sampling-rate modes; image acquisition is a medium-sampling-rate mode; and soil moisture is a low-sampling-rate mode. All modal features are unified to 64 time steps through linear interpolation / mean pooling.

[0124] In this embodiment, the calibrated data is periodically detected, and the autocorrelation coefficient is judged by a 30-second window. A value >0.5 indicates strong periodicity. Strong periodic data is subjected to 64-point FFT, and weak periodic data is subjected to 32-point FFT. The first 30% of the main frequency features are extracted and concatenated with the time domain features to form dual-domain features.

[0125] In this embodiment, the linear feature enhancement branch is: linear layer → Sigmoid activation → intramodal attention weight multiplication; where the weights... ; Let be the attention weights for modality i; Let be the variance of the feature of mode i.

[0126] In this embodiment, the nonlinear feature enhancement branch is: linear layer → FFN → state-space model SSM → intramodal attention weight multiplication.

[0127] In this embodiment, the signal-to-noise ratio is: ; Signal-to-noise ratio; It is the average absolute value of the signal amplitude; This represents the standard deviation of the noise.

[0128] In this embodiment, the intramodal enhancement feature is: The dimension is 256. For enhanced modal features; Enhance the branch output for linear features; This enhances the branch output for nonlinear features.

[0129] In this embodiment, the preset correlation strengths are: LiDAR-Ultrasound: 0.8, LiDAR-Image: 0.7, Image-Soil Moisture: 0.3, and others: 0.2.

[0130] In this embodiment, cross-modal association features: ; This refers to cross-modal correlation features of modal modes; Intramodal enhancement features for modal modes; Preset correlation strength; This is an intramodal enhancement feature for other modes j.

[0131] In this embodiment, intramodal enhancement features and cross-modal correlation features are spliced ​​together, input into a fully connected layer, and output as a 1024-dimensional final feature sequence for each modality.

[0132] In this embodiment, the credibility of each modality's feature data is calculated. During the training phase, the real-time credibility of each modality is calculated based on error statistics, using the following formula: ; The credibility of the modal; These are the model's predicted values; The true value is used for reasoning; the credibility is calculated based on the uncertainty of historical predictions. ; To account for historical prediction uncertainty, modal features with a confidence level greater than 0.3 are retained, while low-confidence modal features are filtered using a moving average. This yields the filtered features.

[0133] In this embodiment, the Dynamic Time Warping (DTW) algorithm aligns all modal features to 100 time steps.

[0134] In this embodiment, cross-modal attention: ; Let be the correlation strength between modes i and j at time step t; , Let i be the 1024-dimensional feature of mode i and j at time step t.

[0135] The working principle and beneficial effects of the above technical solution are as follows: Firstly, it unifies the multi-source heterogeneous raw data into a matrix format and aligns it by timestamp, solving the inconsistency problem of different types of data in the time dimension. This provides a unified and standardized data foundation for subsequent processing, facilitating comprehensive analysis by the model. Secondly, it calculates and calibrates non-stationary indices, reducing non-stationary interference in the data, improving data stability and reliability, and enabling the model to better learn effective information from the data. Thirdly, by configuring modal multi-scale windows and sampling operations, it unifies the features of each modality to a 64-time-step window, eliminating the impact of sampling rate differences, ensuring consistency of different modal data in the time scale, and enhancing the comparability and fusion effect of features. Fourthly, it uses fast Fourier transforms with different numbers of points for modal data with different periodicity, extracting the amplitude and phase features of the main frequency and concatenating them with time-domain features. This fully utilizes the information in the time and frequency domains, enriching feature expression and providing a more comprehensive feature representation for subsequent analysis. The improved MambaFFN module dynamically weights and fuses features based on signal-to-noise ratio through linear and nonlinear feature enhancement branches. This adaptively enhances features according to their quality, highlighting effective information in each modality and improving the quality of individual modality features. A cross-modal association prior matrix is ​​constructed, and each modality feature is weighted and summed according to association strength. This uncovers potential associations between different modalities, enabling the model to comprehensively consider the mutual influence between modalities and improving the overall expressive power of the features. The credibility of each modality feature data is calculated and filtered, removing features with low credibility, reducing noise and redundant information interference, and improving feature quality and model generalization ability. A dynamic time warping algorithm is used to align feature lengths and construct a cross-modal attention matrix to fuse features according to association strength, further optimizing the fusion effect of multimodal features. This allows the model to more accurately capture the associations and changes between different modalities, resulting in more representative and effective multimodal fused features.

[0136] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or high-voltage switchgear that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or high-voltage switchgear.

[0137] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A field robot adaptive navigation path planning system based on convolutional neural networks, characterized in that, include: The twin modeling unit is configured to construct a digital twin model that reflects the dynamic changes of the field environment in real time based on multi-dimensional data of the field environment; The environmental perception unit is configured to construct a convolutional neural network for environmental perception based on historical data. The image data of the field environment is used as input, and the features in the image are automatically extracted based on the convolutional neural network. The output is the recognition results of obstacles, crop areas and passable areas in the environment. The adaptive path planning unit is configured to perform path planning based on environmental information output by the digital twin model and recognition results output by the convolutional neural network, using a fuzzy logic algorithm combined with the kinematic and dynamic constraints of the field robot and path factors. The data acquisition unit is configured to install multiple sensors on the field robot, including: lidar for accurately measuring obstacle distances and terrain heights; ultrasonic sensors for detecting obstacles at close range; and soil moisture sensors for monitoring soil moisture distribution. The data processing unit is configured to use multimodal fusion technology to fuse image data, lidar data, ultrasonic data and soil moisture data to generate multimodal fused data; The data processing unit includes: The acquisition sub-unit is used to acquire multi-source heterogeneous raw data; The preprocessing subunit is used to convert the various types of raw data in the multi-source heterogeneous raw data into a matrix format of timestamp-feature dimension, and to align all data in time dimension based on timestamp; calculate the non-stationary index corresponding to each modality data, perform non-stationary calibration based on the calibration formula of modality affine parameters, and output the calibrated data; configure modal multi-scale windows based on the sampling rate differences of each modality, and unify the multi-scale features of all modalities into 64 time step windows through upsampling linear interpolation and downsampling mean pooling operations to obtain multi-scale unified features; The first fusion subunit is used to perform adaptive time-frequency decomposition on the strongly periodic modes in the calibrated data. The strongly periodic data uses a 64-point fast Fourier transform and the weakly periodic data uses a 32-point fast Fourier transform to extract the amplitude and phase features of the main frequency. These features are then concatenated with the time-domain features to form dual-domain features. The dual-domain features are then fused with the multi-scale unified features to obtain a standardized multi-scale feature sequence. The enhancement subunit is used to perform standardized multi-scale feature processing on each mode through an intra-modal feature enhancement branch composed of an improved MambaFFN module. The improved MambaFFN module includes a linear feature enhancement branch and a nonlinear feature enhancement branch. The outputs of the two branches are dynamically weighted and fused according to the modal feature signal-to-noise ratio to obtain the intra-modal enhanced features. The second fusion subunit is used to construct a cross-modal association prior matrix. The matrix elements are the preset association strengths between each modality. The intra-modal enhancement features of each modality are weighted and summed with the intra-modal enhancement features of other modalities according to the association strength to obtain the cross-modal association features. The third fusion subunit is used to concatenate the intra-modal enhancement features and cross-modal correlation features of each modality, input them into the modal fully connected layer, unify the feature dimension to 1024 dimensions, and output the final feature sequence of each modality; The filtering subunit is used to calculate the credibility of each modal feature data, and to filter each modal feature based on the credibility to obtain the filtered features; The fourth fusion subunit is used to align all filtered features to a uniform length using a dynamic time warping algorithm, construct a cross-modal attention matrix, calculate the correlation strength between any two modal features at each time step, and fuse all modal features by weighting the correlation strength to obtain multimodal fusion features. The preprocessing subunit calculates the non-stationary index corresponding to each modality of data. The non-stationary index is different for different data, including but not limited to: LiDAR data, which is variance plus trend term slope; the trend term slope is calculated by linear regression through a 10-second sliding window; soil moisture data, which is variance plus periodic index; the periodic index is calculated by FFT main frequency amplitude through a 60-second window; ultrasonic data, which is variance plus instantaneous fluctuation amplitude; the instantaneous fluctuation amplitude is the standard deviation of a 50ms window; and image target feature data, which is variance plus target area change rate; the target area change rate is the relative change between adjacent frames. LiDAR data calibration formula: ;in, The average value over a 10-second sliding window; The standard deviation is for a 10-second sliding window. The trend slope; This refers to the current time point; , For learnable affine parameters; This is the calibrated lidar data; This is the raw lidar data; Soil moisture data calibration formula: ;in, Main frequency, , These are the periodic amplitude and phase, respectively; , For learnable affine parameters; For calibrated soil moisture data; This is the raw soil moisture data; The mean of the sliding window; The standard deviation of the sliding window; Ultrasonic data calibration formula: ;in, This refers to the instantaneous fluctuation amplitude; This is a threshold value used to determine whether instantaneous fluctuations need to be compensated. , For learnable affine parameters; The calibrated ultrasonic data; This is the raw ultrasound data; The mean of the sliding window; The standard deviation of the sliding window; Image data calibration formula: ;in, The target area change rate; , For learnable affine parameters; For calibrated image feature data; This refers to the original image feature data; The mean of the sliding window; The standard deviation of the sliding window; The target area change rate, For time, it is used to compensate for changes in area over time; Ultrasonic and lidar are high sampling rate modes; image acquisition is a medium sampling rate mode; soil moisture is a low sampling rate mode. All modal features are unified to 64 time steps through linear interpolation / mean pooling. Periodicity was detected on the calibrated data, and the autocorrelation coefficient was judged by a 30-second window. A value >0.5 indicated strong periodicity. A 64-point FFT was used for the strong periodic data, and a 32-point FFT was used for the weak periodic data. The top 30% of the main frequency features were extracted and concatenated with the time domain features to form a dual-domain feature. Linear feature enhancement branch: Linear layer → Sigmoid activation → Intramodal attention weight multiplication; where the weights ; Let be the attention weights for modality i; Let be the variance of the features of mode i; Nonlinear feature enhancement branch: Linear layer → FFN → State-space model SSM → Intramodal attention weight multiplication; Signal-to-noise ratio: ; Signal-to-noise ratio; It is the average absolute value of the signal amplitude; The standard deviation of the noise; Intramodal enhancement features: The dimension is 256. For enhanced modal features; Enhance the branch output for linear features; Enhance the branch output for nonlinear features; Preset correlation strength: LiDAR-Ultrasound: 0.8, LiDAR-Image: 0.7, Image-Soil Moisture: 0.3, Other: 0.2; Cross-modal association features: ; This refers to cross-modal correlation features of modal modes; For modal intramodal enhancement features; Preset correlation strength; For intra-modal enhancement features of other modes j; The intramodal enhancement features and cross-modal association features are concatenated, input into a fully connected layer, and output as a 1024-dimensional final feature sequence for each modality. Calculate the reliability of feature data for each modality. During the training phase: calculate the real-time reliability of each modality based on error statistics, using the following formula: ; The credibility of the modal; These are the model's predicted values; The true value is used for reasoning; the credibility is calculated based on the uncertainty of historical predictions. ; To account for historical prediction uncertainty, modal features with a confidence level greater than 0.3 are retained, while low-confidence modal features are filtered using a moving average. Dynamic Time Warping (DTW) algorithm aligns all modal features to 100 time steps; Cross-modal attention: ; Let be the correlation strength between modes i and j at time step t; , For modes i and j at time step t, there are 1024-dimensional features. The adaptive path planning unit includes: The thread creation module is configured to use convolutional neural networks to identify and label key entities in the field environment, assigning a unique identifier to each entity. Within the digital twin model, a digital thread link is created for each key entity, recording and updating its state information in real time, including but not limited to position, morphology, speed, and direction, and synchronizing this information to the corresponding digital thread link. Historical state data for each key entity is acquired, including but not limited to changes in position, morphology, and speed, to analyze its movement patterns and trends. A prediction model is built using a long short-term memory network to predict its possible states in the near future, including: using wind speed and direction data combined with the physical characteristics of crops to predict the amplitude and direction of crop swaying; analyzing the movement trajectory and speed of suspected animal areas to predict their movement probability and direction; and outputting the prediction results in structured data form, including but not limited to information on the position and morphological change range of each key entity at future points in time, providing environmental auxiliary data for path planning. Among them, the prediction model built using long short-term memory networks includes: Obtain the model training dataset; the model training dataset includes LiDAR data, ultrasonic data, soil moisture data, and image data; extract features from the training dataset and fuse the extracted features to obtain multimodal fusion features; construct a hierarchical LSTM architecture; the bottom layer is a 2-layer bidirectional LSTM with temporal residual connections, which inputs the final fused features to capture short-term fine-grained temporal dependencies and outputs the bottom layer features; the middle layer performs adaptive time-frequency decomposition on the bottom layer features, constructs time-frequency attention weights to dynamically fuse temporal and frequency domain features, and outputs the middle layer fused features; the top layer extracts shared features through a 1-layer fully connected layer and constructs dual prediction branches; The crop sway prediction branch takes shared features and prior physical characteristics of crops as inputs, and outputs sway amplitude and horizontal direction angle; the animal movement prediction branch takes shared features and suspected animal region features as inputs, and outputs movement probability, horizontal movement direction, and movement speed; a multi-dimensional loss function is constructed, which includes multi-task weighted loss and modality consistency loss; the model is iteratively trained based on the multi-dimensional loss function until the training results meet the requirements, and the prediction model is obtained. The architecture consists of three layers: a bottom layer for feature capture, a middle layer for time-frequency fusion, and an upper layer for multi-task prediction. The bottom layer is an LSTM layer for capturing short-term temporal features, using two bidirectional LSTM layers with a hidden layer dimension of 1024 and a dropout rate of 0.2, for multimodal feature fusion. Captures short-term, fine-grained temporal dependencies and outputs underlying features. Simultaneously, a temporal residual connection is added, and the input features are element-wise added to the LSTM output features: ; The feature vector output by the underlying LSTM; It is a 2-layer bidirectional LSTM network; This serves as the final feature input for multimodal fusion; Temporal features are derived from the underlying output. Preserving temporal dynamics; frequency domain features are extracted by adaptive time-frequency decomposition of the underlying features, extracting frequency domain amplitude and phase features, and compressing the dimension to 1024 through global average pooling; time-frequency attention weights are used to balance the contributions of time domain and frequency domain features. ; For time-frequency attention weights; This is the temporal feature vector output by the underlying LSTM; It is a frequency domain eigenvector; ;in, For frequency domain characteristics, For the Sigmoid activation function, A linear layer that outputs scalar features; a fused feature layer that outputs mid-level features. The autocorrelation coefficient of the sequence is calculated to determine the strength of time dependence; The upper layer includes a multi-task prediction head, which outputs shared features through a fully connected layer, ReLU activation, and batch normalization. The dual prediction branches include a crop swing prediction branch and an animal movement prediction branch; the crop swing prediction branch will share features. The data is concatenated with prior knowledge of crop physical characteristics and used as input features. Passed through two fully connected layers, the output consists of the crop's swaying amplitude and horizontal angle. The animal movement prediction branch will share these features. The features of the crop sway prediction branch and the suspected animal region are concatenated and used as input features. These are then passed through two fully connected layers, and the outputs are the movement probability, horizontal movement direction, and movement speed. A learnable interaction matrix is ​​added between the fully connected layers of the crop sway prediction branch and the animal movement prediction branch to constrain the hidden layer features of both branches. ,in, This is the balance coefficient; A learnable interaction matrix; The hidden layer features of the crop swinging branch after interaction; Features of the hidden layer of the original crop swing branch; Hidden layer features for predicting animal movement branches; Multi-task weighted loss function ; For multi-task weighted loss function; Losses due to crop swaying; Loss prediction for animal movement; crop swaying loss: using MSE loss, with MSE for swaying amplitude and angular period loss for horizontal angle: ; in, Losses due to crop swaying; To predict the horizontal direction angle; This is the true horizontal direction angle; The number of time steps; The predicted swing amplitude; This represents the actual swing amplitude; Animal movement loss: movement probability is calculated using binary cross-entropy, movement direction is calculated using angular periodicity loss, and movement speed is calculated using MSE. ; Predict total losses for animal movement; For binary cross-entropy loss; To predict the probability of movement; This represents the actual movement probability; This refers to the periodic loss of the angle in the horizontal movement direction. To predict the horizontal movement direction angle; This represents the actual horizontal movement direction angle; To predict movement speed; This represents the actual movement speed. Modal consistency loss: Calculate the Pearson correlation coefficient of the mean vectors of each modality feature. ;in, The number of modes; (⋅) represents the mean of the feature dimension; For the preset correlation matrix; This represents the modal consistency loss. (⋅) represents the Pearson correlation coefficient; This represents the total number of time steps. Multidimensional loss function: ; For multi-dimensional loss; Weighted loss for multiple tasks; This represents the modal consistency loss.

2. The adaptive navigation path planning system for field robots based on convolutional neural networks according to claim 1, characterized in that, The adaptive path planning unit further includes: The path adjustment module is configured to obtain the current task type of the field robot and evaluate the priority of the task based on its importance and urgency. Based on current environmental information and task requirements, the target weights of the path planning are dynamically adjusted to meet diverse field operation needs. During the route planning process, changes in the field environment are monitored in real time, and environmental information is quickly updated through a digital twin model; Based on the updated environmental information, the task requirements and path planning objective weights are reassessed, and the path planning strategy is dynamically adjusted.

3. The adaptive navigation path planning system for field robots based on convolutional neural networks according to claim 2, characterized in that, The adaptive path planning unit further includes: The intelligent decision-making module is configured to define a balance mechanism between automated decision-making and human intervention, and to divide tasks into three priorities: high, medium, and low, based on their complexity, risk, and importance. When making automated decisions, high-priority tasks should be prioritized for human intervention, while low-priority tasks should be prioritized for automated decision-making. Create a three-tiered anomaly handling strategy, adopting different handling methods based on the severity and complexity of the anomaly.

4. The adaptive navigation path planning system for field robots based on convolutional neural networks according to claim 2, characterized in that, Based on current environmental information and task requirements, dynamically adjust the target weights of the path planning, including: Based on the task requirements, define multiple target weights for path planning, including: path shortestness weight, smoothness weight, and safety weight. Real-time acquisition of environmental information from the digital twin model to assess the impact of the current environment on path planning objectives; Define fuzzy logic rules and convert environmental information into fuzzy sets; Based on fuzzy logic rules, the rules corresponding to the current environmental information and task requirements are matched, and reasoning calculations are performed to obtain fuzzy reasoning results. The fuzzy inference results are converted into adjustment values ​​for the target weights, and the target weights are dynamically adjusted.

5. The adaptive navigation path planning system for field robots based on convolutional neural networks according to claim 2, characterized in that, The adaptive path planning unit further includes: The lifecycle management module is configured to automatically create a digital thread link when a new critical entity is detected. For digital thread links that have not been updated for a long time, have left the area of ​​interest, or whose entity status has stabilized, delete them from the archived database to free up computing resources.

6. The adaptive navigation path planning system for field robots based on convolutional neural networks according to claim 1, characterized in that, The environmental sensing unit includes: The multi-scale feature extraction module is configured to extract both local and global features from an image simultaneously, and introduces an attention mechanism to enable the convolutional neural network to automatically focus on key regions in the image.

Images