Method, system and readable storage medium for predicting behavioral intention of moving target

Abstract

The application relates to a behavior intention prediction method and system for a moving target and a readable storage medium. The method comprises the following steps: inputting multi-modal to-be-predicted data into a trained multi-line model to obtain a behavior intention prediction result; a training process of the multi-line model comprises the following steps: constructing a target loss function according to a first loss function of a primary task, a second loss function of a secondary task and a preset alignment loss function between single-line models in an initial multi-line model; the primary task is direction-level intention prediction, and the secondary task is behavior-level intention prediction; and the trained multi-line model is obtained based on multi-modal training data and the target loss function, thereby solving the problem of poor behavior intention prediction performance of single-modal data, processing each modal data through the multi-line model, and strengthening the model performance through a multi-task and multi-modal alignment loss function in the training stage, so that the reliability and adaptability of the model are significantly improved.

Description

Technical Field

[0001] This application relates to the field of behavior recognition technology, and in particular to a method, system, and readable storage medium for predicting the behavioral intentions of a moving target. Background Technology

[0002] Predicting the behavioral intent of a moving target involves analyzing its current and historical states (such as position, speed, attitude, and trajectory) to infer its potential future actions or behavioral intentions. Examples include whether a pedestrian will cross the road, whether a vehicle is preparing to change lanes, or whether an aircraft will alter its trajectory. Accurate prediction of behavioral intent is crucial for applications such as intelligent monitoring and intelligent navigation, and is a prerequisite for achieving safe, efficient, and autonomous decision-making.

[0003] Currently, most mainstream behavioral intent prediction methods are based on single-modal data modeling, such as relying on only one information source: visual images, radar point clouds, or GPS trajectories. However, single-modal methods often fail to comprehensively and robustly characterize target behavior in complex dynamic environments: radar has limited detection capabilities when facing "low-altitude, slow-moving, and small" targets; radio monitoring is susceptible to interference from complex electromagnetic environments, leading to signal distortion or loss; and visible light imaging is heavily dependent on lighting conditions, typically only effective during the day or in well-lit environments. Therefore, existing technologies still have significant shortcomings in generalization ability and reliability in complex, variable, or adverse environments.

[0004] There is currently no effective solution to the problem of poor performance in predicting behavioral intent based on single-modal data in related technologies. Summary of the Invention

[0005] This embodiment provides a method, system, and readable storage medium for predicting the behavioral intent of a moving target, in order to solve the problem of poor performance in behavioral intent prediction based on single-modal data in related technologies.

[0006] In a first aspect, this embodiment provides a method for predicting the behavioral intent of a moving target, the method comprising:

[0007] Acquire multimodal data to be predicted for a moving target; input the multimodal data to be predicted into a fully trained multiline model to obtain behavioral intention prediction results;

[0008] The process of obtaining a fully trained multiline model is as follows:

[0009] Acquire the multimodal training data of the moving target;

[0010] A target loss function is constructed based on the first loss function of the first-level task, the second loss function of the second-level task, and the alignment loss function between single-line models in the preset initial multi-line model; the first-level task and the second-level task are obtained by dividing the moving target into behavioral intention predictions, the first-level task is directional intention prediction, and the second-level task is behavioral intention prediction.

[0011] Based on the multimodal training data and the target loss function, the initial multiline model is trained to obtain a fully trained multiline model.

[0012] In some of these embodiments, the first loss function is obtained based on the sum of the first sub-loss functions of each of the single-line models;

[0013] The first sub-loss function is obtained based on the cross-entropy of the first-level task.

[0014] In some of these embodiments, the second loss function is obtained based on the sum of the second sub-loss functions of each of the single-line models;

[0015] The second sub-loss function is obtained based on the cross-entropy of the second-level task under the label data constraint of the first-level task.

[0016] In some of these embodiments, the alignment loss function includes a third loss function;

[0017] The third loss function is obtained based on the similarity of the deep features output by each of the single-line models.

[0018] In some embodiments, the alignment loss function further includes a fourth loss function; the fourth loss function is used to add to the third loss function.

[0019] The fourth loss function is obtained based on the cross-entropy of the secondary task output by each of the single-line models.

[0020] In some embodiments, the single-line model in the initial multi-line model includes: a shallow feature extraction module, a deep feature extraction module, a first-level task prediction module, and a second-level task prediction module;

[0021] The shallow feature extraction module is used to extract initial shallow features from the single-modal data of the multimodal training data;

[0022] The first-level task prediction module is used to generate prediction results for the first-level task based on the initial shallow features.

[0023] The deep feature extraction module is used to obtain initial deep features based on the initial shallow feature extraction;

[0024] The secondary task prediction module is used to generate prediction results for the secondary task based on the initial deep features.

[0025] In some embodiments, the input layer of the single-line model is set according to the modality type of its input data;

[0026] The feature output layer of the single-line model is augmented with a mapping unit to align the output deep features.

[0027] In some embodiments, the multimodal data to be predicted is input into a fully trained multiline model to obtain behavioral intent prediction results, including:

[0028] Each modal data in the multimodal data to be predicted is input into the corresponding single-line model in the fully trained multi-line model to perform single-modal behavioral intention prediction and obtain single-modal target prediction results.

[0029] The prediction results of each single-modal target are fused in a lightweight manner to obtain the behavioral intention prediction result.

[0030] Secondly, this embodiment provides a system for predicting the behavioral intentions of a moving target, including: a data acquisition unit and a computing unit;

[0031] The data acquisition unit includes distributed multimodal sensors for detecting moving targets and generating data to be predicted;

[0032] The computing unit is connected to the data acquisition unit and is used to implement the steps of the method described in any one of the first aspects.

[0033] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the method for predicting the behavioral intent of a moving target as described in the first aspect.

[0034] Compared with related technologies, the method, apparatus, and computer device for predicting the behavioral intent of a moving target provided in this embodiment obtain multimodal data to be predicted from the moving target; input the multimodal data to be predicted into a fully trained multi-line model to obtain the behavioral intent prediction result; wherein, the process of obtaining the fully trained multi-line model is as follows: obtaining multimodal training data of the moving target; constructing a target loss function based on the first loss function of the first-level task, the second loss function of the second-level task, and the alignment loss function between single-line models in the preset initial multi-line model; the first-level task and the second-level task are determined by dividing the moving target... The target's behavioral intent is predicted, with the first-level task being directional intent prediction and the second-level task being behavioral intent prediction. Based on the multimodal training data and the target loss function, the initial multi-line model is trained to obtain a fully trained multi-line model. This solves the problem of poor behavioral intent prediction performance for single-modal data. By processing each modality of data separately through the multi-line model, and enhancing model performance during the training phase through multi-task and multimodal aligned loss functions, high-precision and robust behavioral intent prediction is achieved in multimodal scenarios. This effectively copes with environmental interference and significantly improves the reliability and adaptability of the system in complex scenarios.

[0035] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is a hardware structure block diagram of the terminal for the method of predicting the behavioral intent of a moving target in the embodiments of this application;

[0038] Figure 2 This is a flowchart illustrating the training method for the multi-line model in the embodiments of this application;

[0039] Figure 3 This is a schematic diagram of the structure of the multiline model in the embodiments of this application;

[0040] Figure 4 This is a schematic diagram of the behavior intention prediction system for a moving target in an embodiment of this application;

[0041] Figure 5 This is a schematic diagram of a drone behavior intent prediction learning framework based on task hierarchy and distributed multimodal alignment in a preferred embodiment of this application.

[0042] Reference numerals: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 41, data acquisition unit; 42, computing unit; 43, moving target. Detailed Implementation

[0043] To better understand the purpose, technical solution, and advantages of this application, the application is described and explained below in conjunction with the accompanying drawings and embodiments.

[0044] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0045] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the motion target behavior intention prediction method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0046] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the motion target behavior intention prediction method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0048] This embodiment provides a method for predicting the behavioral intent of a moving target. The method includes the following steps: acquiring multimodal data to be predicted of the moving target; inputting the multimodal data to be predicted into a fully trained multiline model to obtain the behavioral intent prediction result.

[0049] Specifically, the multimodal data to be predicted includes data captured from the moving target by at least two different modal sensors. A fully trained multi-line model includes at least two single-line models, each processing data acquired by one modal sensor. The single-line model divides the prediction process of the moving target's behavioral intent into a coarse first-level task and a fine second-level task. The first-level task is directional intent prediction, and the second-level task is behavioral intent prediction. Behavioral intent is a further subdivision of the directional intent into specific action intents, which are tailored to the moving target. Taking a drone as an example, directional intents include horizontal movement intent, vertical movement intent, and hovering intent; behavioral intents include further left, right, forward, and backward movements based on horizontal movement intents, further upward or downward movements based on vertical movement intents, and further hovering movements based on hovering intents. The final behavioral intent prediction result output by the fully trained multi-line model includes at least behavioral intents.

[0050] Among them, such as Figure 2 As shown, the process of obtaining a fully trained multiline model includes:

[0051] Step S210: Obtain multimodal training data of the moving target.

[0052] Specifically, multimodal training data includes data from different sensors collected for moving targets in different modalities, along with corresponding label data. The label data includes at least labels for directional intent and labels for behavioral intent.

[0053] Step S220: Construct the target loss function based on the first loss function of the first-level task, the second loss function of the second-level task, and the alignment loss function between single-line models in the preset initial multi-line model; the first-level task and the second-level task are obtained by dividing the moving target into behavioral intention predictions, the first-level task is directional intention prediction, and the second-level task is behavioral intention prediction.

[0054] Specifically, at least two sensors with different modalities are deployed in a distributed manner to capture data of moving targets. The captured data, after being labeled, can be used as multimodal training data and input into the corresponding single-line model for feature extraction and behavior prediction. Each single-line model includes a shallow feature extraction module and a deep feature extraction module. The shallow feature extraction module extracts shallow features from the data and performs directional intent prediction based on these shallow features, i.e., the first-level task. The deep feature extraction module further extracts features from the shallow features obtained by the shallow feature extraction module to obtain deep features, which are used for behavior-level intent prediction, i.e., the second-level task.

[0055] The first loss function can be calculated based on the difference between the directional intent prediction result and the directional intent label of the single-line model; the second loss function can be calculated based on the difference between the behavioral intent prediction result and the behavioral intent label of the single-line model; the alignment loss function can be calculated based on the differences in deep features and / or the differences in behavioral intent prediction results among the single-line models. A multi-task balancing factor is introduced into the objective loss function, assigning corresponding weights to the first, second, and alignment loss functions to achieve a good balance between different prediction and alignment tasks. The shallow prediction of the first-level task adds intermediate constraints to the model, ensuring that shallow parameters can also participate fully and effectively in learning. The multi-level task joint objective loss function forces the model to learn complementary information at different levels, improving the model's performance in processing multimodal data, while also making the optimization path smoother and more likely to converge to a better local optimum.

[0056] Step S230: Based on the multimodal training data and the target loss function, train the initial multiline model to obtain the fully trained multiline model.

[0057] Specifically, this embodiment does not limit the selection of feature extraction networks in the initial multi-line model. For example, taking the ResNet residual network as an example, the output of the first ResNet-block is defined as the shallow features, and the final output features of ResNet are defined as the deep features. Each single-line model can be the same or different, and the input and output layers of the model can be modified according to the data modality.

[0058] In this embodiment, multimodal data to be predicted of moving targets is acquired; the multimodal data to be predicted is input into a fully trained multi-line model to obtain behavioral intention prediction results. The process of obtaining a fully trained multi-line model is as follows: acquiring multimodal training data of moving targets; constructing a target loss function based on the first loss function of the first-level task, the second loss function of the second-level task, and the alignment loss function between single-line models in the preset initial multi-line model; the first-level task and the second-level task are obtained by dividing the behavioral intention prediction of moving targets, the first-level task is directional intention prediction, and the second-level task is behavioral intention prediction; based on the multimodal training data and the target loss function, the initial multi-line model is trained to obtain a fully trained multi-line model, which solves the problem of poor behavioral intention prediction performance of single-modal data. By processing each modality of data separately through the multi-line model, and enhancing the model performance during the training phase through multi-task and multimodal alignment loss functions, high-precision and high-robust behavioral intention prediction is achieved in multimodal scenarios, effectively coping with environmental interference and significantly improving the reliability and adaptability of the system in complex scenarios.

[0059] In some of these embodiments, the first loss function is obtained by summing the first sub-loss functions of each single-line model; the first sub-loss function is obtained based on the cross-entropy of the first-level task.

[0060] Specifically, taking a multi-line model comprising two single-line models S and T as an example, the formula for calculating the first loss function L1 is as follows:

[0061] ;

[0062] in, This is the first sub-loss function of the linear model S. This is the first sub-loss function of the single-line model T. Specifically, i∈[1,2,3], meaning the first-level task is treated as three independent recognition tasks (horizontal movement, vertical movement, and hovering); y i It is the corresponding first-level task tag, p i S This is the prediction result for the first-level task corresponding to the single-line model S; p i T It is the prediction result of the first-level task corresponding to the single-line model T.

[0063] In this embodiment, the model is efficiently guided to optimize the first-level task based on the cross-entropy function of each single-line model, thereby accelerating subsequent deep training.

[0064] In some of these embodiments, the second loss function is obtained by summing the second sub-loss functions of each single-line model; the second sub-loss function is obtained by cross-entropy of the second-level task under the label data constraint of the first-level task.

[0065] Specifically, the formula for calculating the second loss function L2 is as follows:

[0066] ;

[0067] in, This is the second sub-loss function of the linear model S. This is the second sub-loss function of the single-line model T. Specifically, each first-level task i has a corresponding second-level task k (for example: when i=1, k∈[1,2,3,4], that is, horizontal movement intention is divided into four movement intentions: forward, backward, left, and right; when i=2, k∈[1,2], vertical movement intention is divided into two movement intentions: upward and downward; when i=3, k=1, the second-level task under the hovering intention is one hovering intention), z k It is a secondary task tag, p k S This is the prediction result for the secondary task corresponding to the single-line model S; p k TThis is the prediction result of the second-level task corresponding to the single-line model T. Let the prediction result of the second-level task be compared with the label data of the first-level task (i.e., the label y of the first-level task). i Multiply by , so that if the first-level task does not occur, it is not included in the loss calculation, thus achieving a constraint effect on the second-level task.

[0068] In this embodiment, the cross-entropy functions of the second-level tasks of each single-line model are combined to further optimize the second-level tasks based on the first-level tasks, thereby improving the prediction accuracy of multimodal models.

[0069] In some embodiments, the alignment loss function includes a third loss function; the third loss function is obtained based on the similarity of the deep features output by each single-line model.

[0070] Specifically, the final output features of the feature extraction network in the single-line model are used as deep features. First, the deep features F output by each single-line model are calculated. S With F T The similarity between Sim(F) S ,F T This quantifies the similarity of feature vectors in multimodal data and determines the potential relationships between different features. The similarity calculation method in this embodiment is as follows:

[0071] ;

[0072] Where n is the vector length of the deep feature. The formula for calculating the third loss function L(Feature) is as follows:

[0073] .

[0074] Therefore, in one implementation, the target loss function is obtained by weighting the first loss function, the second loss function, and the third loss function.

[0075] In this embodiment, feature alignment loss further enhances the consistency of multimodal data features, thereby extracting richer semantic information from the multimodal features. Features from different modalities complement and corroborate each other, providing a more comprehensive perspective for accurately predicting behavioral intentions and helping to improve the model's generalization ability and robustness. When facing various complex situations and different environmental conditions, integrating multimodal information through deep features can make the model more stable and reliable, reduce the possibility of misjudgment, and provide more reliable auxiliary decision-making.

[0076] In some embodiments, the alignment loss function further includes a fourth loss function; the fourth loss function is added to the third loss function; the fourth loss function is obtained based on the cross-entropy of the secondary tasks output by each single-line model.

[0077] Specifically, for the behavior-level intent prediction results (prediction results of the secondary task) of the deep feature extraction module, the cross-entropy loss function is used to measure the difference between prediction results of different modalities. The prediction results R of the two single-line models are then compared. S With R T Alignment:

[0078] ;

[0079] Based on this, in one implementation, the target loss function is obtained by weighting the first loss function L1, the second loss function L2, the third loss function L(Feature), and the fourth loss function L(Result):

[0080] ;

[0081] Here, α, β, and λ are multi-task balancing factors used to balance the weights of each task in the model's regression process, ensuring a good balance between the different tasks. In practice, if a particular task is more important to the final prediction result, its corresponding balancing factor can be appropriately increased to ensure the model focuses more on that task.

[0082] In this embodiment, information from different modalities has different characteristics and advantages. By aligning the prediction results of multiple modalities, the advantages of each modality can be fully utilized, and comprehensive consideration can be made to improve the accuracy and comprehensiveness of the prediction. Even if an error or interference occurs in a single modality, the results of other modalities can play a role in correction and supplementation, reducing the risk of misjudgment, thereby improving the coordination and reliability of the entire prediction system.

[0083] In some of these embodiments, see Figure 3 The initial multi-line model includes multiple single-line models. Each single-line model includes: a shallow feature extraction module, a deep feature extraction module, a first-level task prediction module, and a second-level task prediction module.

[0084] The shallow feature extraction module is used to extract initial shallow features from the single-modal data of the multimodal training data.

[0085] The Level 1 Task Prediction Module is used to generate prediction results for Level 1 tasks based on initial shallow features.

[0086] The deep feature extraction module is used to obtain initial deep features based on the initial shallow feature extraction.

[0087] The secondary task prediction module is used to generate prediction results for secondary tasks based on the initial deep features.

[0088] In this embodiment, the shallow feature extraction module extracts shallow features from the data and performs directional intent prediction based on these shallow features to obtain the prediction result for the first-level task. The deep feature extraction module further extracts features from the shallow features obtained by the shallow feature extraction module, obtaining deep features used for behavior-level intent prediction to obtain the prediction result for the second-level task. This achieves a coarse-to-fine action segmentation of behavior, effectively avoiding confusion between different intents, enabling the model to more accurately distinguish behaviors, and thus significantly improving the model's prediction accuracy.

[0089] In some embodiments, the input layer of the single-line model is configured according to the modality type of its input data; the feature output layer of the single-line model is augmented with mapping units to align the output deep features.

[0090] In this embodiment, the single-line model can set different network structures according to the type of distributed sensors accessed. The input layer and input layer can be adaptively adjusted, and the input layer includes the output layer of the deep feature extraction module.

[0091] In some embodiments, multimodal data to be predicted is input into a fully trained multiline model to obtain behavioral intent prediction results, including:

[0092] Step S310: Input the modal data of each modality in the multimodal data to be predicted into the corresponding single-line model in the trained multi-line model to perform single-modal behavioral intention prediction and obtain the single-modal target prediction result.

[0093] Step S320: Lightweight fusion of the prediction results of each single-modal target is performed to obtain the behavioral intention prediction result.

[0094] Specifically, each single-line model outputs a prediction result independently, resulting in multiple predictions. Lightweight post-processing, such as averaging and weighted voting, is then applied to these multiple predictions to obtain a unified output. For example, single-line model S scores 0.95 and 0.05 for the possibilities 0 and 1, respectively, while single-line model T scores 0.85 and 0.15 for the same two possibilities. After averaging, the combined scores are 0.9 and 0.1, respectively, resulting in a final result of 0.

[0095] In this embodiment, semantic alignment is achieved during the training phase, and high-precision, robust behavioral intent prediction can be obtained simply by fusing the information during inference. This method takes into account both modal characteristics and the advantages of multi-source complementarity, efficiently fusing multimodal information and solving the uncertainty in the prediction process.

[0096] This embodiment also provides a system for predicting the behavioral intent of a moving target, which includes a data acquisition unit and a computing unit. Figure 4As shown, the data acquisition unit 41 includes distributed multimodal sensors for detecting moving targets 43 and generating data to be predicted; the moving targets 43 include, but are not limited to, animals or objects in a motion scene. The computing unit 42, connected to the data acquisition unit 41, is used to implement the steps of the moving target behavior intention prediction method in any of the above embodiments.

[0097] In this embodiment, behavioral intention prediction results are obtained by inputting multimodal data to be predicted into a fully trained multi-line model. The process of obtaining the fully trained multi-line model is as follows: acquiring multimodal training data of moving targets; constructing a target loss function based on the first loss function of the first-level task, the second loss function of the second-level task, and the alignment loss function between single-line models in the preset initial multi-line model; the first-level task and the second-level task are obtained by dividing the behavioral intention prediction of moving targets, with the first-level task being directional intention prediction and the second-level task being behavioral intention prediction; training the initial multi-line model based on the multimodal training data and the target loss function to obtain a fully trained multi-line model, which solves the problem of poor behavioral intention prediction performance of single-modal data. By processing each modality of data separately through the multi-line model, and enhancing the model performance during the training phase through multi-task and multimodal alignment loss functions, high-precision and high-robust behavioral intention prediction is achieved in multimodal scenarios, effectively coping with environmental interference and significantly improving the reliability and adaptability of the system in complex scenarios.

[0098] The present embodiment will now be described and illustrated through preferred embodiments.

[0099] Figure 5 This is a schematic diagram of a UAV behavior intention prediction learning framework based on task hierarchy and distributed multimodal alignment, according to a preferred embodiment of this invention. Unmanned systems such as UAVs, due to their unique advantages in cost and maneuverability, can be applied to many dangerous and challenging tasks such as tactical reconnaissance and electronic warfare. However, current methods for predicting the motion intention of UAVs are limited to analyzing data from a single modality, resulting in poor accuracy. This preferred embodiment proposes a motion intention prediction framework applicable to multimodal systems. This framework overcomes the shortcomings of multimodal models in feature and result alignment, helping to improve the intelligence level of decision-making systems. Figure 5 As shown, the moving target is a drone, and the process of predicting the drone's behavioral intentions includes:

[0100] 1. Define the hierarchical prediction task: Divide the UAV behavior intention prediction process into a primary task and a secondary task. The primary task is directional intention prediction, including horizontal movement intention, vertical movement intention, and hovering intention. The secondary task is behavioral intention prediction, which further predicts left, right, forward, and backward movements based on horizontal movement intention; ascending or descending movements based on vertical movement intention; and continuing to hover movements based on hovering intention.

[0101] 2. Constructing the Initial Multi-Line Model: The initial multi-line model consists of two initial single-line models, S and T. These two models are connected to two distributed sensors of different modalities (radar and optical sensors), respectively, to achieve data capture, feature extraction, and behavior prediction for the UAV. Each single-line model includes a shallow feature extraction module, a deep feature extraction module, a first-level task prediction module, and a second-level task prediction module. The shallow feature extraction module extracts initial shallow features. The first-level task prediction module generates prediction results for the first-level task based on the initial shallow features. The deep feature extraction module extracts initial deep features based on the initial shallow features. The second-level task prediction module generates prediction results for the second-level task based on the initial deep features.

[0102] 3. Construct the target loss function, train the initial multiline model, and obtain the fully trained multiline model: The formula for the target loss function L(Total) is as follows:

[0103] ;

[0104] Where L1 is the first loss function, L2 is the second loss function, L(Feature) is the third loss function, and L(Result) is the fourth loss function.

[0105] The formula for calculating the first loss function L1 is as follows:

[0106] ;

[0107] Where i∈[1,2,3], the first-level task is regarded as three independent recognition tasks (horizontal movement, vertical movement, and hovering); y i It is the corresponding first-level task tag, p i S This is the prediction result for the first-level task corresponding to the single-line model S; p i T It is the prediction result of the first-level task corresponding to the single-line model T.

[0108] The formula for calculating the second loss function L2 is as follows:

[0109] ;

[0110] Specifically, each primary task i has a corresponding secondary task k (for example: when i=1, k∈[1,2,3,4], meaning horizontal movement intentions are divided into four movement intentions: forward, backward, left, and right; when i=2, k∈[1,2], vertical movement intentions are divided into two movement intentions: upward and downward; when i=3, k=1, hovering intentions are the secondary task of hovering). k It is a secondary task tag, p k S This is the prediction result for the secondary task corresponding to the single-line model S; p k T Let y be the prediction result of the second-level task corresponding to the single-line model T, and let the prediction result of the second-level task be compared with the label y of the first-level task. i Multiply them to ensure that the loss is not included in the calculation if the first-level task does not occur, thus constraining the second-level tasks.

[0111] The formula for calculating the third loss function L(Feature) is as follows:

[0112] ;

[0113] Sim(F S ,F T F represents the deep features output by each linear model. S With F T The similarity between them is calculated using the following formula:

[0114] ;

[0115] Where n is the vector length of the deep feature.

[0116] The formula for calculating the fourth loss function L(Result) is as follows:

[0117] ;

[0118] Among them, R S With R T These are the prediction results for deep features from two single-line models.

[0119] α, β, and λ are multi-task balancing factors used to balance the weights of each task in the model's regression process, ensuring a good balance between the model and different tasks. In practice, the balancing factor corresponding to a specific loss term in the objective loss function can be adaptively adjusted to ensure the model focuses more on that loss task.

[0120] Based on multimodal training data and the target loss function, an initial multiline model is trained to obtain a fully trained multiline model. This preferred embodiment does not limit the selection of the feature extraction network in the initial multiline model. For example, using ResNet as an example, the output of the first ResNet-block is defined as the shallow features, and the final output features of ResNet are defined as the deep features.

[0121] 4. UAV Behavior Intent Prediction: Acquire multimodal data to be predicted from the UAV. This multimodal data is obtained by detecting moving targets using radar and optical sensors. Each modality of the multimodal data is input into the corresponding single-line model within a fully trained multi-line model to perform single-modal behavior intent prediction, yielding the single-modal target prediction result. Lightweight fusion of these single-modal target prediction results is then performed to obtain the final behavior intent prediction result.

[0122] The advantages of this preferred embodiment include:

[0123] 1. The task of predicting the behavior intent of drones is divided into two levels: the first level is directional prediction and the second level is behavior prediction. This more detailed classification of drone behavior can effectively avoid confusion between different intents, enabling the model to distinguish behaviors more accurately and thus significantly improving the prediction accuracy of the model.

[0124] 2. Cosine similarity is used to calculate the similarity between deep feature vectors of multimodal data, and a feature alignment loss is proposed to ensure the consistency of distributed multimodal features and tightly integrate information from different data sources. This captures target motion information from different dimensions, avoiding the limitations of a single data source, and successfully extracts richer semantic information from multimodal features, thereby more accurately predicting the target's behavioral intent. Furthermore, the framework provided in this preferred embodiment has high generalization ability, suitable for combinations of two to more lines, and conveniently adapts to more modal data.

[0125] 3. The cross-entropy loss function is used to align the behavioral-level prediction results of the multi-line model, so that the prediction results of each modality cooperate and work together. During training, the coordination between modalities and the reliability of the final prediction can be greatly improved. During prediction, data from a single modality can be used for prediction, which is more flexible.

[0126] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0127] Furthermore, in conjunction with the motion target behavior intention prediction method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the motion target behavior intention prediction methods in the above embodiments.

[0128] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0129] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0130] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0131] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for predicting the behavioral intention of a moving target, characterized in that, The method includes: Acquire multimodal data to be predicted for a moving target; input the multimodal data to be predicted into a fully trained multiline model to obtain behavioral intention prediction results; The process of obtaining a fully trained multiline model is as follows: Acquire the multimodal training data of the moving target; A target loss function is constructed based on the first loss function of the first-level task, the second loss function of the second-level task, and the alignment loss function between single-line models in the preset initial multi-line model; the first-level task and the second-level task are obtained by dividing the moving target into behavioral intention predictions, the first-level task is directional intention prediction, and the second-level task is behavioral intention prediction. Based on the multimodal training data and the target loss function, the initial multiline model is trained to obtain a fully trained multiline model.

2. The method for predicting the behavioral intent of a moving target according to claim 1, characterized in that, The first loss function is obtained by summing the first sub-loss functions of each of the single-line models; The first sub-loss function is obtained based on the cross-entropy of the first-level task.

3. The method for predicting the behavioral intent of a moving target according to claim 1, characterized in that, The second loss function is obtained by summing the second sub-loss functions of each of the single-line models; The second sub-loss function is obtained based on the cross-entropy of the second-level task under the label data constraint of the first-level task.

4. The method for predicting the behavioral intent of a moving target according to claim 1, characterized in that, Alignment loss function, including third loss function; The third loss function is obtained based on the similarity of the deep features output by each of the single-line models.

5. The method for predicting the behavioral intent of a moving target according to claim 4, characterized in that, The alignment loss function also includes a fourth loss function; the fourth loss function is used to add to the third loss function. The fourth loss function is obtained based on the cross-entropy of the secondary task output by each of the single-line models.

6. The method for predicting the behavioral intent of a moving target according to claim 1, characterized in that, The single-line model in the initial multi-line model includes: a shallow feature extraction module, a deep feature extraction module, a first-level task prediction module, and a second-level task prediction module; The shallow feature extraction module is used to extract initial shallow features from the single-modal data of the multimodal training data; The first-level task prediction module is used to generate prediction results for the first-level task based on the initial shallow features. The deep feature extraction module is used to obtain initial deep features based on the initial shallow feature extraction; The secondary task prediction module is used to generate prediction results for the secondary task based on the initial deep features.

7. The method for predicting the behavioral intent of a moving target according to claim 6, characterized in that, The input layer of the single-line model is set according to the modal type of its input data; The feature output layer of the single-line model is augmented with a mapping unit to align the output deep features.

8. The method for predicting the behavioral intent of a moving target according to claim 1, characterized in that, The multimodal data to be predicted is input into a fully trained multiline model to obtain behavioral intention prediction results, including: Each modal data in the multimodal data to be predicted is input into the corresponding single-line model in the fully trained multi-line model to perform single-modal behavioral intention prediction and obtain single-modal target prediction results. The prediction results of each single-modal target are fused in a lightweight manner to obtain the behavioral intention prediction result.

9. A system for predicting the behavioral intentions of a moving target, characterized in that, include: Data acquisition unit and computing unit; The data acquisition unit includes distributed multimodal sensors for detecting moving targets and generating data to be predicted; The computing unit is connected to the data acquisition unit and is used to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images