An intelligent duty system based on multimodal large model

Abstract

The present invention relates to the field of intelligent duty technology, and specifically, to an intelligent duty system based on a multimodal large model. It includes a sensor acquisition unit: used to deploy various physical sensors and software sensors to collect multimodal information in the environment; a data storage processing unit: used to store and pre-process the raw data collected by the sensor acquisition unit; a data analysis unit: used to analyze the processed data and identify abnormal behavior patterns; a behavior analysis prediction unit: used to determine whether the current behavior is abnormal and predict future behavior trends based on the data of the data storage processing unit and the abnormal behavior pattern; a decision-making alarm unit: used to alarm and promptly notify staff of the identified abnormal behavior. In this intelligent duty system based on a multimodal large model, it is not only possible to evaluate the current situation in real time, but also to predict future behavior trends.

Description

Technical Field

[0001] The present invention relates to the field of intelligent duty technology, and in particular to an intelligent duty system based on a multi-modal large model. Background Art

[0002] In today's society, with the acceleration of urbanization and the rapid development of science and technology, public safety, traffic management, environmental protection and other fields are facing unprecedented challenges. Traditional monitoring and management systems often rely on a single information source, such as video surveillance, environmental monitoring equipment, etc., which limits the system's perception range and analysis capabilities, and it is difficult to cope with the increasingly complex urban environment and social needs. In this context, the intelligent duty system based on multimodal large models came into being and became one of the key technologies to improve the level of intelligent urban management. The intelligent duty system based on multimodal large models can integrate multimodal information from different sensors and data sources to achieve deep fusion and cross-validation of data. This integration of multimodal data not only expands the system's perception capabilities, but also can capture the full picture of events from multiple angles and levels, and improve the accuracy and comprehensiveness of event analysis. In the existing technology, the intelligent duty system often focuses on the response to the current state and lacks the prediction function for the future, which limits its role in preventing security incidents and optimizing operation management. Therefore, an intelligent duty system based on multimodal large models is designed. Summary of the invention

[0003] The purpose of the present invention is to provide an intelligent duty system based on a multimodal large model to solve the problem that the above-mentioned background technology often focuses on the response to the current state and lacks the prediction function for the future.

[0004] To achieve the above object, the present invention provides an intelligent duty system based on a multimodal large model, comprising:

[0005] Sensor acquisition unit: used to deploy physical sensors and software sensors to collect multimodal information in the environment;

[0006] Data storage and processing unit: used to store and pre-process the raw data collected by the sensor acquisition unit;

[0007] Data analysis unit: used to analyze the processed data and identify abnormal behavior patterns;

[0008] Among them, the data analysis unit includes a trajectory construction module, a positioning estimation module and a pattern recognition module;

[0009] Behavior analysis and prediction unit: used to determine whether the current behavior is abnormal and predict future behavior trends based on the data in the data storage and processing unit and the abnormal behavior pattern;

[0010] Among them, the behavior analysis and prediction unit includes a behavior modeling module and a prediction analysis module;

[0011] Decision-making alarm unit: used to alarm and promptly notify staff of abnormal behaviors identified in the data analysis unit and behavior analysis and prediction unit.

[0012] As a further improvement of the present technical solution, in the sensor acquisition unit, the multimodal information includes video data, environmental parameter data, and the real-time location point and movement path of a person or object.

[0013] As a further improvement of the technical solution, in the data analysis unit:

[0014] The trajectory construction module is used to connect continuous time series location point data to form the movement trajectory of people or objects;

[0015] The positioning estimation module is used to optimize the trajectory data through the Kalman filter algorithm to reduce the noise caused by sensor errors or environmental contamination;

[0016] The pattern recognition module is used to analyze trajectory data and identify abnormal behavior patterns of people or objects.

[0017] As a further improvement of the technical solution, the specific steps of the trajectory construction module are as follows:

[0018] Compute the distance between two points:

[0019]

[0020]

[0021]

[0022] in, is an intermediate variable; ; is the difference in latitude between two points; is the latitude of the first point; is the latitude of the second point; ; is the difference in longitude between two points; is the longitude of the first point; is the longitude of the second point; is the radian of the central angle between two points; is the great circle distance between two points; is the average radius of the Earth;

[0023] Estimate missing values ​​using linear interpolation:

[0024]

[0025] in, is the predicted latitude of the point to be interpolated; is the longitude corresponding to the interpolation point;

[0026] Using the time sequence, all position points are sorted in chronological order, each position point is regarded as a node on the trajectory, and adjacent nodes are connected with line segments to form a moving trajectory.

[0027] As a further improvement of the technical solution, in the positioning estimation module, the trajectory data is optimized by the Kalman filter algorithm as follows:

[0028] Predicted state estimation:

[0029]

[0030] in, The current state estimate predicted by the previous state; is the state transfer matrix; is the state estimate of the previous moment; is the control input matrix; is the control vector;

[0031] Prediction covariance:

[0032]

[0033] in, is the predicted covariance matrix; is the modified covariance matrix of the previous moment; is the transpose operation; is the process noise covariance matrix;

[0034] Kalman gain:

[0035]

[0036] in, is the Kalman gain; is the observation model matrix; is the observation noise covariance matrix;

[0037] Status Update:

[0038]

[0039] in, is the updated state estimate; is the actual observed value at the current moment;

[0040] Covariance update:

[0041]

[0042] in, is the updated covariance matrix; is the identity matrix.

[0043] As a further improvement of the technical solution, in the pattern recognition module, the abnormal behavior pattern of a person or object includes a regular path, a stop point analysis, an abnormal speed change, and a direction change;

[0044] Among them, conventional path analysis identifies high-density areas by calculating the distance between trajectory points to form path clusters; stay point analysis determines whether the time difference between consecutive location points exceeds the threshold based on timestamps and identifies the stay time.

[0045] As a further improvement of the technical solution, the distance between trajectory points is calculated to identify high-density areas and form path clusters, as follows:

[0046] S1, identify high-density trajectory segments according to the distance threshold and the minimum number of points, map the trajectory points to the feature space, build a hierarchical clustering tree of trajectory segments, and merge paths according to similarity;

[0047] S2, analyze the time difference between trajectory points, identify the area of ​​long-term stay, and calculate the horizontal speed and direction of each trajectory segment as clustering features;

[0048] S3, identify frequently occurring trajectory patterns, build a transition probability model from one location to another, and identify normal paths;

[0049] S4. Based on clustering results and pattern recognition, paths with higher frequencies are extracted as regular paths.

[0050] As a further improvement of the technical solution, in the behavior analysis prediction unit:

[0051] The behavior modeling module is used to establish a behavior model based on historical trajectory data and pattern recognition results to determine whether the current behavior of people and objects is normal;

[0052] The predictive analysis module is used to predict future movement paths, stop points and the probability of abnormal behavior based on the behavior model through data mining methods.

[0053] As a further improvement of the technical solution, in the behavior modeling module, the specific method of establishing the behavior model is as follows:

[0054] ;

[0055] in, In accordance with the strategy When taking action, from the state Initial potential risk assessment value; For a given state Take an action The probability of ; The behavioral state of a person or object; ; is the set of possible trajectory changes; The action is executed in the current behavior state s After that, transition to the next state probability; is the rationality function; is the discount factor; In accordance with the strategy When taking action, from the state Begin with a potential risk assessment.

[0056] As a further improvement of the technical solution, the specific method of predicting the future movement path, possible stop points or the probability of abnormal behavior based on the behavior model through the data mining method is as follows:

[0057] Future mobility path:

[0058] ;

[0059] in, In accordance with the strategy When taking action, from the state Initial potential risk assessment value; In accordance with the strategy When taking action, from the state Initial potential risk assessment value;

[0060] Possible stops:

[0061]

[0062] in, For the status The probability of choosing the stay action; For the status The probability of staying down;

[0063] Probability of abnormal behavior:

[0064]

[0065] in, For the status Next action Probability of being considered abnormal; For the status Take Probability of action;

[0066] exist The temperature factor is introduced for optimization, and the optimized expression is:

[0067]

[0068] in, For the optimized state Take Probability of action; For Action The value function of For Action The next action; For Action The value function of is the temperature parameter;

[0069]

[0070] in, For the optimized state Next action The probability of being considered an anomaly.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] 1. In this intelligent duty system based on multi-modal large models, through the fine design of trajectory construction, positioning optimization and pattern recognition modules, it can effectively build high-precision whereabouts trajectories, and use advanced technologies such as Kalman filtering to eliminate noise interference and ensure the accuracy of behavior analysis. The pattern can identify basic paths and speeds, greatly improving the sensitivity and specificity of abnormal behavior judgment;

[0073] 2. In this intelligent duty system based on multimodal large models, the behavior modeling and predictive analysis module combines historical data with machine learning algorithms, which can not only evaluate the current situation in real time, but also predict future behavior trends, helping to detect potential security threats or operational problems in advance, and also providing a valuable time window for resource scheduling and emergency response, thereby improving management efficiency and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is the overall flow chart of the present invention;

[0075] Figure 2 It is a system flow chart of the data analysis unit in the present invention;

[0076] Figure 3is a system block diagram of the behavior analysis and prediction unit in the present invention;

[0077] The meaning of each number in the figure is:

[0078] 1. Sensor acquisition unit; 2. Data storage and processing unit; 3. Data analysis unit; 31. Trajectory construction module; 32. Positioning estimation module; 33. Pattern recognition module; 4. Behavior analysis and prediction unit; 41. Behavior modeling module; 42. Prediction analysis module; 5. Decision-making alarm unit. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] Example

[0081] See also Figure 1-3 As shown, an intelligent duty system based on a multimodal large model is provided, comprising: a sensor acquisition unit 1, a data storage and processing unit 2, a data analysis unit 3, a behavior analysis and prediction unit 4 and a decision alarm unit 5;

[0082] The sensor acquisition unit 1 is used to deploy physical sensors and software sensors to collect multimodal information in the environment; the multimodal information includes video data, environmental parameter data, and real-time location points and movement paths of people or objects. Video data can capture visual scenes, environmental parameter data (such as temperature) provides environmental background information, and location point and movement path data record the dynamic changes of entities, which together constitute a three-dimensional description of the environmental state.

[0083] The data storage processing unit 2 is used to store and pre-process the raw data collected by the sensor acquisition unit 1; because the received raw data often includes noise, redundancy or inconsistent formats, the data storage processing unit 2 needs to pre-process the data, including data cleaning, format conversion, denoising, standardization and other operations to improve data quality and provide clean and structured data for subsequent data analysis and machine learning model training;

[0084] The data analysis unit 3 is used to analyze the processed data and identify abnormal behavior patterns;

[0085] The data analysis unit 3 includes a trajectory construction module 31, a positioning estimation module 32 and a pattern recognition module 33;

[0086] Data Analysis Unit 3:

[0087] The trajectory construction module 31 is used to connect continuous time series position point data to form a movement trajectory of a person or object; it mainly connects the scattered time series position point data provided by the sensor acquisition unit 1 to construct a coherent movement trajectory of a person or object, which usually involves time sorting of data, missing value filling and smoothing processing to ensure the continuity and rationality of the trajectory. Through trajectory construction, the system can intuitively display the movement path of the target object, providing a basis for further behavior analysis and prediction.

[0088] The specific steps of the trajectory construction module 31 are as follows:

[0089] Compute the distance between two points:

[0090]

[0091]

[0092]

[0093] in, is an intermediate variable; ; is the difference in latitude between two points; is the latitude of the first point; is the latitude of the second point; ; is the difference in longitude between two points; is the longitude of the first point; is the longitude of the second point; is the radian of the central angle between two points; is the great circle distance between two points; is the average radius of the Earth;

[0094] Estimate missing values ​​using linear interpolation:

[0095]

[0096] in, is the predicted latitude of the point to be interpolated; is the longitude corresponding to the interpolation point;

[0097] By using time sequence, all location points are sorted in chronological order to ensure the consistency and temporal correctness of trajectory construction; each location point is regarded as a node on the trajectory, and scattered location points are converted into an ordered sequence of trajectory nodes; adjacent nodes are connected with line segments to form a moving trajectory, completing the transformation from location point data to a visual trajectory, which is convenient for subsequent behavior analysis and pattern recognition.

[0098] The above steps ensure the conversion from raw location data to a coherent and accurate movement trajectory. The distance between two points is calculated to ensure the geographic accuracy of the trajectory. Linear interpolation makes up for the missing data and maintains the integrity of the trajectory. Time sorting and node connection build the coherence and visibility of the trajectory, laying a solid foundation for subsequent data analysis and pattern recognition.

[0099] The coherent trajectory constructed by the trajectory construction module 31 provides the necessary input for the pattern recognition module 33. Without the trajectory construction step, it will be difficult for the pattern recognition module 33 to identify meaningful behavior patterns from scattered location point data. Through the trajectory construction module 31, the pattern recognition module 33 can more accurately identify the behavior pattern and potential abnormal behavior of the target object based on the complete movement path;

[0100] The positioning estimation module 32 is used to optimize the trajectory data through the Kalman filter algorithm to reduce the noise caused by sensor errors or environmental interference; the positioning estimation module 32 uses algorithms such as Kalman filtering to optimize the trajectory data, which can reduce the noise caused by sensor errors or environmental interference and improve the accuracy of position estimation. Kalman filtering is an effective state estimation method. It can predict the most likely value of the next state based on previous observations and current measurement results, while minimizing the mean square error of the prediction error. By continuously updating the prediction and measurement values, the positioning estimation module 32 can effectively smooth the trajectory, eliminate sudden outliers, and make the trajectory smoother and more accurate, thereby providing high-quality input data for subsequent pattern recognition;

[0101] In the positioning estimation module 32, the trajectory data is optimized by the Kalman filter algorithm as follows:

[0102] Predicted state estimation:

[0103]

[0104] in, The current state estimate predicted by the previous state; is the state transfer matrix; is the state estimate of the previous moment; is the control input matrix; is the control vector;

[0105] Prediction covariance:

[0106]

[0107] in, is the predicted covariance matrix; is the modified covariance matrix of the previous moment; is the transpose operation; is the process noise covariance matrix;

[0108] Kalman gain:

[0109]

[0110] in, is the Kalman gain; is the observation model matrix; is the observation noise covariance matrix;

[0111] Status Update:

[0112]

[0113] in, is the updated state estimate; is the actual observed value at the current moment;

[0114] Covariance update:

[0115]

[0116] in, is the updated covariance matrix; is the identity matrix.

[0117] Through the above steps, the Kalman filter algorithm realizes real-time estimation and update of the system state, and dynamically adjusts the uncertainty of the state estimation. In the positioning estimation module 32, it can effectively deal with factors such as sensor errors and environmental interference. Through the process of prediction, updating and correction, it continuously optimizes the trajectory data and improves the accuracy and stability of position estimation. The Kalman filter algorithm shows strong robustness and accuracy when dealing with dynamic system state estimation problems, especially when facing noisy data and imperfect models.

[0118] The positioning estimation module 32 improves the quality and reliability of the trajectory data by reducing the noise and error in the trajectory, so that the trajectory constructed by the trajectory construction module 31 more truly reflects the actual movement of people or objects, and avoids misleading subsequent analysis due to erroneous trajectories;

[0119] The pattern recognition module 33 is used to analyze trajectory data and identify abnormal behavior patterns of people or objects. In the pattern recognition module 33, the abnormal behavior patterns of people or objects include regular paths, stop point analysis, abnormal speed changes and direction changes; the task of the pattern recognition module 33 is to conduct in-depth analysis of the optimized trajectory data and identify abnormal behavior patterns of people or objects, including but not limited to identifying common movement patterns (such as walking in a straight line, circling, wandering), abnormal behaviors (such as sudden acceleration, deceleration, deviation from regular paths) and periodic behaviors (such as round-trip paths that appear at regular intervals). The pattern recognition module 33 uses machine learning algorithms (such as clustering, classification, sequence pattern mining, etc.) to automatically extract features from a large amount of trajectory data, learn and summarize patterns, and provide key information for the decision-making alarm unit 5 of the intelligent duty system.

[0120] Among them, conventional path analysis identifies high-density areas by calculating the distance between trajectory points to form path clusters; stay point analysis determines whether the time difference between consecutive location points exceeds the threshold based on the timestamp and identifies the stay time; by comparing the time difference between consecutive trajectory points, the area with a longer stay time is identified. This step is based on the assumption that the time difference near the stay point will be significantly greater than the time difference during movement;

[0121] Calculate the distance between trajectory points to identify high-density areas and form path clusters as follows:

[0122] S1, identify high-density trajectory segments according to the distance threshold and the minimum number of points, map the trajectory points to the feature space, build a hierarchical clustering tree of trajectory segments, and merge paths according to similarity;

[0123] The distance threshold and minimum number of points are set to identify high-density areas. The distance threshold determines the maximum distance between two points that are considered close enough, while the minimum number of points defines the minimum number of points required to constitute a high-density area. This helps to filter out sparse trajectory points and focus on frequently passed paths; the geographic location information of the trajectory points is converted into points in the feature space, which may contain multiple information dimensions such as location coordinates and timestamps, which is convenient for subsequent clustering analysis; the hierarchical clustering tree is a way to recursively segment a data set. By calculating the similarity between different trajectory points, similar points or clusters are gradually merged to form a tree structure, which helps to gradually identify high-density areas from local to global, as well as the hierarchical relationship between these areas; based on the similarity measure between trajectory segments (such as location proximity, time consistency, etc.), similar trajectory segments are gradually merged to form coherent path clusters, which helps to identify recurring path patterns, that is, the movement paths of high-density areas;

[0124] S2. Analyze the time difference between trajectory points, identify the areas where long-term stay is achieved, and calculate the horizontal speed and direction of each trajectory segment as clustering features; these features help distinguish different types of movement patterns, such as fast linear movement and slow wandering, so as to better identify movement behavior patterns in high-density areas;

[0125] S3, identify frequently occurring trajectory patterns, build a probability model of transition from one location to another, and identify normal paths; it helps to identify normal paths, that is, those frequently occurring movement patterns that reflect the typical behavior habits of people or objects;

[0126] S4. Based on clustering results and pattern recognition, extract the paths with higher frequency as regular paths. The extraction of regular paths helps to understand the normal behavior patterns of people or objects, and provides a basis for subsequent applications such as anomaly detection and resource optimization.

[0127] Further, the behavior analysis prediction unit 4 is used to determine whether the current behavior is abnormal and predict future behavior trends based on the data in the data storage and processing unit 2 and the abnormal behavior pattern;

[0128] The behavior analysis and prediction unit 4 includes a behavior modeling module 41 and a prediction and analysis module 42; the behavior modeling module 41 is used to establish a behavior model based on historical trajectory data and pattern recognition results to determine whether the current person and object are normal behaviors;

[0129] The prediction analysis module 42 is used to predict the future movement path, stop points and the probability of abnormal behavior based on the behavior model through data mining methods.

[0130] In the behavior modeling module 41, the specific method of establishing the behavior model is as follows:

[0131] ;

[0132] in, In accordance with the strategy When taking action, from the state Initial potential risk assessment value; For a given state Take an action The probability of , reflects the behavioral preference in a specific state; ; The behavior state of a person or object, such as the combined state of a person or object at different locations and time points; ; is a set of possible trajectory changes, such as going left, going right, stopping, etc.; After executing action a in the current behavior state s, it transitions to the next state The probability of , which reflects the dynamic characteristics of the behavior pattern; is the rationality function; A discount factor that reflects the importance of future rewards. In behavioral analysis, it helps balance immediate behavior and long-term planning; In accordance with the strategy When taking action, from the state Begin with a potential risk assessment.

[0133] The formula calculates the Start by following the strategy The expected potential risk assessment value , which provides a quantitative standard for judging whether the current behavior is abnormal. Abnormal behavior often leads to a higher potential risk assessment value;

[0134] By iterative calculation , the model can learn the risk assessment values ​​under different states and action combinations, and then optimize the strategy , making it more consistent with normal behavior patterns;

[0135] Based on historical data and current status, the formula can help predict possible future states and behaviors, including movement paths, stop points, and the probability of abnormal behavior. Implementation reflects the strategy Risk assessment of the next future state;

[0136] Since the strategy in the formula and state transition probability It is dynamically adjusted based on historical data, and the model can adapt to environmental changes, such as the emergence of new behavioral patterns or changes in existing patterns, to maintain the accuracy of predictions.

[0137] Through data mining methods, future movement paths, possible stop points or the probability of abnormal behavior are predicted based on behavioral models. Predicting future movement paths helps analyze the changing trends of traffic flow, predict congestion points, provide a basis for traffic signal control, road construction and public transportation planning, and alleviate traffic pressure; by analyzing the movement patterns of vehicles and pedestrians, possible traffic accident points are predicted, and measures are taken to intervene in advance, such as adding warning signs, adjusting road design, etc., to reduce the accident rate; predicting high-risk areas or stop points can guide law enforcement agencies to adjust patrol routes and frequencies, concentrate resources on locations where problems may occur, and improve the efficiency and pertinence of police deployment; predicting the probability of abnormal behavior can help detect potential threats early, such as predicting riots or terrorist attacks that may be caused by crowd gatherings, so that preventive measures can be taken to avoid or reduce harm.

[0138] The specific method is as follows:

[0139] Future mobility path:

[0140] ;

[0141] in, In accordance with the strategy When taking action, from the state Initial potential risk assessment value; In accordance with the strategy When taking action, from the state Initial potential risk assessment value;

[0142] By maximizing the strategy Expected return under , choose the best action To predict future movement paths, ensuring that the predicted paths are based on the behavior patterns that are most likely to lead to the best results;

[0143] By state transition probability and instant rewards , the model can take into account the impact of the current action on the next state and the benefits or costs of the immediate behavior;

[0144] By discount factor ,The model can balance the potential value of immediate behavior and future states, ensuring that the predicted path not only considers immediate needs but also long-term planning;

[0145] Possible stops:

[0146]

[0147] in, For the status The probability of choosing the stay action; For the status The probability of staying down;

[0148] Probability of abnormal behavior:

[0149]

[0150] in, For the status Next action Probability of being considered abnormal; For the status Take Probability of action;

[0151] exist The temperature factor is introduced for optimization, and the optimized expression is:

[0152]

[0153] in, For the optimized state Take Probability of action; For Action The value function of For Action The next action; For Action The value function of is the temperature parameter;

[0154] Temperature parameters The introduction of enables the model to balance exploration and exploitation when selecting actions, that is, it increases the exploration of unknown or low-probability actions at high temperatures, and tends to select high-value actions at low temperatures;

[0155] The action probability after temperature optimization through exponential function and normalization Smoother, allowing for finer distinctions even between low-value and high-value actions;

[0156]

[0157] in, For the optimized state Next action The probability of being considered an anomaly.

[0158] Optimized abnormal behavior probability It more accurately reflects the degree of behavioral deviation under temperature regulation, which helps to improve the accuracy and response speed of anomaly detection.

[0159] Finally, the decision alarm unit 5 is used to alarm and promptly notify the staff of the abnormal behaviors identified in the data analysis unit 3 and the behavior analysis and prediction unit 4.

[0160] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An intelligent duty system based on a multimodal large model, characterized in that: include: Sensor acquisition unit (1): used to deploy physical sensors and software sensors to collect multimodal information in the environment; A data storage and processing unit (2): used for storing and preprocessing the raw data collected by the sensor collection unit (1); Data analysis unit (3): used to analyze the processed data and identify abnormal behavior patterns; The data analysis unit (3) includes a trajectory construction module (31), a positioning estimation module (32) and a pattern recognition module (33); Behavior analysis and prediction unit (4): used to determine whether the current behavior is abnormal and predict future behavior trends based on the data in the data storage and processing unit (2) and the abnormal behavior pattern; Wherein, the behavior analysis prediction unit (4) includes a behavior modeling module (41) and a prediction analysis module (42); Decision-making alarm unit (5): used to issue an alarm and promptly notify staff members of abnormal behaviors identified by the data analysis unit (3) and the behavior analysis and prediction unit (4); In the behavior analysis prediction unit (4): The behavior modeling module (41) is used to establish a behavior model based on historical trajectory data and pattern recognition results to determine whether the current person and object behavior is normal; The prediction analysis module (42) is used to predict the future movement path, stop point and probability of abnormal behavior based on the behavior model through data mining methods; In the behavior modeling module (41), the specific method of establishing the behavior model is as follows: ； in, In accordance with the strategy When taking action, from the state Initial potential risk assessment value; For a given state Take an action probability; ; The behavioral state of a person or object; ; is the set of possible trajectory changes; After executing action a in the current behavior state s, it transitions to the next state probability; is the rationality function; is the discount factor; In accordance with the strategy When taking action, from the state Initial potential risk assessment value; The specific method of predicting the future movement path, possible stop points or the probability of abnormal behavior based on the behavior model through the data mining method is as follows: Future mobility path: ； in, In accordance with the strategy When taking action, from the state Initial potential risk assessment value; In accordance with the strategy When taking action, from the state Initial potential risk assessment value; Possible stops: ； in, For the status The probability of choosing the stay action; For the status The probability of staying down; Probability of abnormal behavior: ； in, For the status Next action Probability of being considered abnormal; For the status Take Probability of action; exist The temperature factor is introduced for optimization, and the optimized expression is: ； in, For the optimized state Take Probability of action; For Action The value function of For Action The next action; For Action The value function of is the temperature parameter; ； in, For the optimized state Next action The probability of being considered an anomaly.

2. The intelligent duty system based on multimodal large model according to claim 1 is characterized in that: In the sensor acquisition unit (1), the multimodal information includes video data, environmental parameter data, and the real-time location points and movement paths of people or objects.

3. The intelligent duty system based on multimodal large model according to claim 2 is characterized in that: In the data analysis unit (3): The trajectory construction module (31) is used to connect the continuous time series location point data to form the movement trajectory of the person or object; The positioning estimation module (32) is used to optimize the trajectory data through the Kalman filter algorithm to reduce the noise caused by sensor errors or environmental contamination; The pattern recognition module (33) is used to analyze trajectory data and identify abnormal behavior patterns of people or objects.

4. The intelligent duty system based on multimodal large model according to claim 3 is characterized in that: The specific steps of the trajectory construction module (31) are as follows: Compute the distance between two points: ； ； ； in, is an intermediate variable; ; is the difference in latitude between two points; is the latitude of the first point; is the latitude of the second point; ; is the difference in longitude between two points; is the longitude of the first point; is the longitude of the second point; is the radian of the central angle between two points; is the great circle distance between two points; is the average radius of the Earth; Estimate missing values ​​using linear interpolation: ； in, is the predicted latitude of the point to be interpolated; is the longitude corresponding to the interpolation point; Using the time sequence, all position points are sorted in chronological order, each position point is regarded as a node on the trajectory, and adjacent nodes are connected with line segments to form a moving trajectory.

5. The intelligent duty enforcement system based on multimodal large model according to claim 4 is characterized in that: In the positioning estimation module (32), the trajectory data is optimized by the Kalman filter algorithm as follows: Predicted State Estimates: ； in, The current state estimate predicted by the previous state; is the state transfer matrix; is the state estimate of the previous moment; is the control input matrix; is the control vector; Prediction covariance: ； in, is the predicted covariance matrix; is the modified covariance matrix of the previous moment; is the transpose operation; is the process noise covariance matrix; Kalman gain: ； in, is the Kalman gain; is the observation model matrix; is the observation noise covariance matrix; Status Update: ； in, is the updated state estimate; is the actual observed value at the current moment; Covariance update: ； in, is the updated covariance matrix; is the identity matrix.

6. The intelligent duty enforcement system based on multimodal large model according to claim 5 is characterized in that: In the pattern recognition module (33), the abnormal behavior pattern of a person or object includes a regular path, a stop point analysis, an abnormal speed change, and a direction change; Among them, conventional path analysis identifies high-density areas by calculating the distance between trajectory points to form path clusters; stay point analysis determines whether the time difference between consecutive location points exceeds the threshold based on timestamps and identifies the stay time.

7. The intelligent duty enforcement system based on multimodal large model according to claim 6 is characterized in that: The distance between trajectory points is calculated to identify high-density areas and form path clusters, as follows: S1, identify high-density trajectory segments according to the distance threshold and the minimum number of points, map the trajectory points to the feature space, build a hierarchical clustering tree of trajectory segments, and merge paths according to similarity; S2, analyze the time difference between trajectory points, identify the area of ​​long-term stay, and calculate the horizontal speed and direction of each trajectory segment as clustering features; S3, identify frequently occurring trajectory patterns, build a transition probability model from one location to another, and identify normal paths; S4. Based on clustering results and pattern recognition, paths with higher frequencies are extracted as regular paths.

Images