Visual analysis method and device for complex operation

Abstract

The invention belongs to the technical field of operation scene analysis, and particularly relates to a visual analysis method and device for complex operation. According to the method, grid division is carried out on an operation scene according to original track points of a complex operation process in a space dimension and a time dimension, the original track points are mapped to corresponding grids, and then a neural network model is pre-trained by using minimized reconstruction loss; and then joint training is carried out on the pre-trained neural network model by using the joint loss fusing the minimum reconstruction loss and the clustering loss, so that feature extraction is carried out on the mapped spatial-temporal trajectory data, the extracted optimized feature vectors are clustered to obtain each trajectory cluster, the clustering result is more accurate, and the clustering efficiency is improved. The accuracy of work area division is improved, the accuracy of work type judgment is improved, and the evaluation and analysis accuracy of complex work is higher.

Description

Technical Field

[0001] The present invention belongs to the technical field of operation scene analysis, and specifically relates to a visual analysis method and device for complex operations. Background Art

[0002] Complex tasks involve multiple important spatial regions, with numerous agents performing complex tasks across these regions. This generates massive amounts of spatiotemporal trajectory data, which exhibits multimodal and mixed characteristics. To comprehensively analyze complex task scenarios, efficiently utilize important spatial regions, and optimize task processes and spatial layouts, multiscale assessment and visual analysis of complex tasks is required. This allows users to explore the spatiotemporal distribution of the same task and different tasks, and comprehensively analyzes spatiotemporal utilization. Without loss of generality, assuming the input is raw trajectories acquired from a tracker, their timing and velocity information can be estimated. Stable spatial flows exist in complex task scenarios. These flows are characterized by clusters of geometrically similar trajectories. Each spatial flow represents repetitive activities connecting subspaces with specific functions, thereby revealing certain semantic information. All spatial flows in complex task scenarios collectively form a sound foundation for describing scene activities. This foundation, known as intrinsic invariants, captures data heterogeneity in the presence of noise and randomness, enabling accurate analysis of complex tasks.

[0003] However, computing this foundation presents numerous challenges. Because it contains many spatial flows, it is highly heterogeneous and multimodal. Direct statistical analysis of trajectories, such as using simple descriptive statistics like average speed, results in averaging of useful information, insufficient descriptive information, and failure to capture data heterogeneity. Given that these spatial flows are formed by geometrically similar clusters of trajectories, some researchers have proposed spatiotemporal clustering of trajectory points to obtain spatial flows, thereby understanding the spatiotemporal distribution of tasks within the scene. This lays the foundation for subsequent analysis of the spatiotemporal usage of the scene and the analysis of anomalous trajectories or tasks. Spatiotemporal clustering typically relies on point-matching similarity metrics to compare trajectories. A common approach is to use similarity metrics such as the longest common subsequence (LCSS), edit distance (EDR), and dynamic time warping (DTW) to perform similarity matching, followed by clustering using clustering algorithms such as K-Means. However, since the original trajectories in complex operation scenarios are massive and complex data with a high degree of spatiotemporal coupling, directly clustering the original trajectories will lead to low clustering accuracy. The operation type determined based on the clustering results will also have low accuracy, which in turn leads to low accuracy in the evaluation and analysis of complex operations. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and device for visual analysis of complex operations, so as to solve the problem of low accuracy of complex operation analysis in the prior art and effectively improve the accuracy and reliability of complex operation analysis.

[0005] In order to solve the above technical problems, the present invention provides a visual analysis method for complex operations, including: obtaining original trajectory data of the complex operation process, and gridding the operation scene according to the original trajectory points in the spatial dimension and the temporal dimension respectively; mapping the original trajectory points to the corresponding grids to obtain mapped spatiotemporal trajectory data; inputting the mapped spatiotemporal trajectory data into a joint model to obtain trajectory clusters; the joint model includes a clustering model and an optimized neural network model, the optimized neural network model is first pre-trained by minimizing the reconstruction loss, and then jointly trained with the pre-trained neural network model by using a joint loss that combines the minimization of the reconstruction loss and the clustering loss, the optimized neural network model is used to extract optimized feature vectors, and the clustering model is used to cluster the extracted optimized feature vectors to obtain trajectory clusters; determining the operation type corresponding to each trajectory cluster, and analyzing the complex operation based on the operation type.

[0006] Furthermore, the clustering loss includes intra-cluster compactness loss, inter-cluster separation loss and balanced regularization loss; wherein, the intra-cluster compactness loss is used to describe the similarity of samples within a trajectory cluster, the inter-cluster separation loss is used to describe the separability between trajectory clusters, and the balanced regularization loss is used to describe the size deviation of trajectory clusters.

[0007] Furthermore, the process of gridding the operation scene according to the original trajectory points is to first use the density clustering algorithm to cluster the original trajectory points from the spatial dimension and the temporal dimension respectively, and then grid the operation scene based on the density clustering results. The denser the distribution area of ​​the trajectory points, the smaller the corresponding grid.

[0008] Furthermore, the process of determining the job type corresponding to each trajectory cluster is to determine the access sequence of the trajectory cluster and determine the job type corresponding to the trajectory cluster based on the access sequence; the process of determining the access sequence of the trajectory cluster is to judge whether each trajectory point in the trajectory cluster meets the effective contact event triggering condition, and obtain the access sequence of the trajectory cluster based on the satisfied trajectory points; wherein, the effective contact event triggering condition is: the distance between the trajectory point and the job-related position is less than the job radius and the continuous stay time exceeds the minimum time threshold required for job execution.

[0009] Furthermore, the process of determining the job type corresponding to the trajectory cluster is to pre-define the access sequence template corresponding to each job type, and match the access sequence with the access sequence template; if the match is successful, the job type corresponding to the successfully matched access sequence template is used as the job type of the trajectory cluster; if the match is not successful, manual review is used to determine whether it is an abnormal trajectory. If it is not an abnormal trajectory, it is determined whether the access sequence is a valid access sequence. If it is a valid sequence, the job type of the access sequence is marked and added to the access sequence template.

[0010] Furthermore, the reconstruction loss is minimized as:

[0011] L r =-logp(y|x)

[0012] Among them, L r To minimize the reconstruction loss, y is the original trajectory in the dataset, x is the interference trajectory generated based on y, and p is the probability.

[0013] Furthermore, the analysis of complex jobs based on job types includes displaying the spatiotemporal distribution of all jobs under complex tasks, detailed information of each job, job trajectories, and abnormal trajectories.

[0014] Furthermore, the analysis of complex jobs based on job types also includes determining the spatiotemporal usage indicators of each job, and the spatiotemporal usage indicators include at least two types of data: regional density, regional out-degree, regional in-degree, regional betweenness centrality, edge weight, and edge betweenness centrality.

[0015] The beneficial effects of the above technical solution are as follows: the present invention is an improved invention creation, which uses a joint model to determine each trajectory cluster. The processing of the joint model includes three steps: pre-training, feature optimization (joint training) and clustering. The neural network model is first pre-trained by minimizing the reconstruction loss, and then the pre-trained neural network model is further trained using a loss function that combines the minimization of the reconstruction loss and the clustering loss, so that the model can meet the clustering task requirements while optimizing the trajectory feature representation and extract a better feature vector. Finally, the optimized feature vectors output by the neural network are clustered, and finally each trajectory cluster is accurately generated to complete the effective division of the spatiotemporal trajectory data. If the clustering accuracy is improved, then the trajectory cluster obtained based on the clustering result is more accurate, which improves the accuracy of job type judgment and makes the evaluation and analysis of complex jobs more accurate.

[0016] In order to solve the above technical problems, the present invention further provides a visual analysis device for complex operations, including a processor, wherein the processor adopts the above-mentioned visual analysis method for complex operations to perform visual analysis of complex operations.

[0017] Furthermore, the device also includes a user interface, which is used to visualize the data of complex tasks, including displaying the spatiotemporal distribution of all jobs under complex tasks, detailed information of each job, job trajectory, abnormal trajectory and spatiotemporal usage indicators of each job, and the spatiotemporal usage indicators include at least two types of data: regional density, regional out-degree, regional in-degree, regional betweenness centrality, edge weight and edge betweenness centrality.

[0018] The beneficial effects of the above technical solution are as follows: the present invention is an improved invention creation, which uses a joint model to determine each trajectory cluster. The processing of the joint model includes three steps: pre-training, feature optimization (joint training) and clustering. The neural network model is first pre-trained by minimizing the reconstruction loss, and then the pre-trained neural network model is further trained using a loss function that combines the minimization of the reconstruction loss and the clustering loss, so that the model can meet the clustering task requirements while optimizing the trajectory feature representation and extract a better feature vector. Finally, the optimized feature vectors output by the neural network are clustered, and finally each trajectory cluster is accurately generated to complete the effective division of the spatiotemporal trajectory data. If the clustering accuracy is improved, then the trajectory cluster obtained based on the clustering result is more accurate, which improves the accuracy of job type judgment and makes the evaluation and analysis of complex jobs more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the visual analysis principle of complex operations in the embodiment of the method of the present invention;

[0020] Figure 2 It is a user interface diagram of an implementation method of the present invention. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present invention more clear, the specific embodiments of the present invention are further described below with reference to the accompanying drawings.

[0022] Without specifying the number of clusters, the present invention uses a loss function that combines minimization of reconstruction loss and clustering loss to optimize the neural network model pre-trained by minimizing reconstruction loss, thereby improving the accuracy of clustering and making the job type determined according to the clustering results more accurate.

[0023] Method implementation

[0024] The present invention provides a visual analysis method for complex operations. The implementation principle of the method is as follows: Figure 1 As shown, the following steps are included:

[0025] 1. Feature Encoding

[0026] 1) Obtain the original trajectory data of the complex operation process and divide the operation scene into grids from the spatial dimension and time dimension respectively.

[0027] The original spatiotemporal trajectory data consists of a series of spatiotemporal points. In one embodiment, the operation scene can be grid-divided in the spatial dimension and the temporal dimension according to the original trajectory data (i.e., the original trajectory points) in a uniform manner, and then the original trajectory points are mapped to the corresponding grids. Then, based on the mapped grid sequence, the feature representation of the trajectory is learned to further divide the different operations in the scene.

[0028] However, in real life, the trajectory data of complex operations are uneven in space and time distribution, and often have different densities in different areas and different time periods. Using a uniform grid for data mapping will result in a low accuracy rate in determining the operation scene, which in turn leads to a low accuracy rate in the evaluation and analysis of complex operations. Therefore, as a preferred embodiment, the present invention performs space division of the operation scene according to the density of the original trajectory points in the trajectory dataset (Trajectories) in the spatial dimension and the temporal dimension respectively. The greater the density of the original trajectory point distribution area, the smaller the divided grid, so that the area with dense trajectory point distribution is divided into finer small grids, and the area with sparse trajectory is divided into larger grids, so that the trajectory points mapped to the grid are more realistic. By dynamically adjusting the size of the grid in the time and space dimensions, the acquired trajectory timing characteristics are more accurate. Specifically, a density clustering algorithm is used to cluster the original trajectory points in the spatial and temporal dimensions, obtaining clustering results with different densities. The work scene is then gridded based on the density clustering results, dividing work areas with dense trajectory points into smaller and more refined tokens, while work areas with sparse trajectory points can be divided into larger tokens. Tokens are generated to adapt to the uneven distribution of trajectory data in the spatial and temporal dimensions, and each grid is encoded. The DBSCAN algorithm can be used for density clustering, followed by a density-driven adaptive partitioning method to adaptively partition dense and sparse areas in the spatial dimension. In the temporal dimension, the time interval is adjusted to a dynamic size to more accurately represent the temporal characteristics of the trajectory.

[0029] 2) Map the original trajectory points to the corresponding grid to obtain the mapped spatiotemporal trajectory data.

[0030] This paper considers both the spatial and temporal dimensions of trajectory points, using a natural language processing (NLP) model to convert them into discrete units called tokens. These discrete spatiotemporal units form a set similar to a vocabulary in NLP tasks, mapping the original trajectory points to corresponding grids. Ultimately, the original trajectory data is represented as a set of spatiotemporal token sequences with spatial and temporal characteristics, providing more suitable input for subsequent deep learning models.

[0031] 2. Deep Clustering Based on Feature Enhancement

[0032] The present invention constructs a joint model, which includes a clustering model and an optimized neural network model. The construction process is as follows: the neural network model is pre-trained using a minimized reconstruction loss, then the pre-trained model is co-trained using a joint loss that combines the minimized reconstruction loss and clustering loss to obtain an optimized neural network model. The optimized neural network model obtained after pre-training and co-training is used to extract optimized feature vectors, which are then clustered to accurately generate trajectory clusters.

[0033] 1) Pre-train the neural network model.

[0034] Neural network models are used for feature learning and are pretrained by minimizing the reconstruction loss. The mapped spatiotemporal trajectory data is sequence data, namely the spatiotemporal token sequence defined in step 1. Therefore, the neural network model can use the Seq2Seq model, which handles sequence-to-sequence mapping tasks, to learn the feature representation of the entire spatiotemporal trajectory data. Examples include RNN-based Seq2Seq models, LSTM-based Seq2Seq models, and Transformer models. Taking the Seq2Seq model as an example, it is built on an autoencoder architecture and consists of an encoder and a decoder. The encoder maps the spatiotemporal trajectory data into a low-dimensional feature space, while the decoder is responsible for reconstructing these low-dimensional features back into the original trajectory data space, effectively extracting the core features of the data and achieving dimensionality reduction. Minimizing the reconstruction loss refers to the loss in the neural network model's reconstruction of the spatiotemporal token sequence (i.e., the mapped spatiotemporal trajectory data). Specifically, the encoder reads the input spatiotemporal trajectory sequence sequentially and updates the hidden state at each time step. During this process, the hidden state at each time point depends not only on the current input but also on the hidden state at the previous moment, with both jointly determining the current hidden state. Ultimately, the hidden state output by the encoder is a fixed-dimensional vector that effectively represents the global characteristics of the input data. The decoder predicts the symbol of the next spatiotemporal point based on the hidden state output by the encoder. Specifically, the decoder starts from an initial state and gradually generates outputs related to the input trajectory sequence. During decoding, the decoder's hidden state is updated at each time step based on the previous hidden state and output. Through the encoding and decoding process, the Seq2Seq model can effectively capture the temporal characteristics of spatiotemporal trajectories.

[0035] The minimized reconstruction loss function of the neural network model is expressed using negative log-likelihood loss:

[0036] L r =-logp(y|x)

[0037] Among them, L r To minimize the reconstruction loss, y is the original trajectory in the dataset, x is the interference trajectory generated based on y, such as adding noise or randomly discarding some trajectory points, and p is the probability. When the neural network uses the Seq2Seq model, the neural network loss is the reconstruction loss.

[0038] 2) Joint training is performed on the pre-trained neural network model.

[0039] The present invention does not directly use the representation learned by the neural network for clustering. Instead, it combines the clustering process with trajectory representation learning for joint training: a joint loss function that combines the minimization of reconstruction loss and clustering loss is used to jointly train the pre-trained neural network model to obtain an optimized neural network model. This allows the optimized neural network model to extract optimized feature vectors, which are more suitable for cluster representation, thereby obtaining higher quality clustering results. The joint loss function is:

[0040] L=λL r +(1-λ)L c

[0041] Among them, L is the joint loss, L r is the clustering loss, and λ∈[0,1] is a hyperparameter for the balance and loss. The clustering loss includes intra-cluster compactness loss, inter-cluster separation loss, and balance regularization loss. The cluster structure is defined by the relationship between samples (rather than fixed cluster centers), without the need to pre-set the number of clusters.

[0042] Among them, the intra-cluster closeness loss directly defines the similarity of samples within a cluster and is the basis of clustering. It promotes the closeness of samples within the same cluster in the embedding space and is the core driving force for cluster formation. The intra-cluster closeness loss function is:

[0043]

[0044] Among them, L com is the intra-cluster density loss, N k (i) is the k nearest neighbors of the i-th sample, z i is the embedding representation of the i-th sample, z i is the embedding representation of the jth sample, and N is the size of the dataset.

[0045] The inter-cluster separation loss directly defines the separation between clusters, thereby avoiding cluster overlap and ensuring that samples between different clusters maintain a certain distance. Avoiding inter-cluster overlap is the key to forming clear cluster boundaries. The inter-cluster separation loss function is:

[0046]

[0047] Among them, L sep is the inter-cluster separation loss, N mk (i) is the m*k nearest neighbors of the i-th sample, δ is the interval threshold, and this loss function can keep the sample away from non-adjacent samples.

[0048] The balanced regularization loss does not directly define the cluster structure, but improves overall stability by balancing cluster sizes. It is used to describe the deviation of trajectory cluster sizes and balances cluster sizes by maximizing entropy, preventing some clusters from being too large or too small, and assisting in optimizing the overall cluster structure. The balanced regularization loss function is:

[0049]

[0050] Among them, L bal To balance the regularization loss, n c is the number of samples in the cth potential cluster.

[0051] 3) Input the extracted optimized feature vector into any clustering model to obtain each trajectory cluster.

[0052] The optimized feature vector is input into any clustering model of the joint model to obtain clustering results (Cluster-1, Cluster-2, ... Cluster-k). Each cluster represents a trajectory cluster, and each trajectory cluster represents a job task.

[0053] 3. Task Type Recognition

[0054] After obtaining the clustering results, we perform task identification on the trajectory clusters represented by each clustering result to determine the corresponding job type. This paper uses a hybrid analysis method that integrates domain knowledge and lightweight data mining to achieve classification and labeling of task types. The specific process is as follows:

[0055] 1) Determine the access sequence of trajectory clusters.

[0056] The continuous agent movement trajectory is converted into a discrete sequence of spatial location access symbols, and the behavioral characteristics are explicitly expressed through spatial semantic mapping. Specifically, based on the spatial location of the trajectory data and the job, the trigger conditions for valid contact events are defined: the distance between the trajectory point and the job-related location is less than the job radius and the continuous stay time exceeds the minimum time threshold required for job execution. Each trajectory point in the trajectory cluster is judged to determine whether it meets the valid contact event trigger conditions. When the trigger conditions are met, it is determined to be a valid contact event. Through this method, the original trajectory cluster is symbolically encoded to form an access sequence with clear job semantics.

[0057] 2) Build a two-layer rule matching mechanism to identify job tasks for access sequences.

[0058] The first layer predefines access sequence templates for each task. Access sequences are matched against these templates. If a match is successful, the job type corresponding to the matching access sequence template is used as the job type for the trajectory cluster. Specifically, an expert-defined task pattern knowledge base is used to quickly label high-frequency tasks using predefined typical access sequence templates. This knowledge base, extracted from historical task records, contains standardized sequence patterns for multiple typical job processes.

[0059] The second layer targets unmatched abnormal sequences. If the match is not successful, the unmatched access sequence is manually reviewed to determine whether it is an abnormal trajectory. If it is not an abnormal trajectory, it is determined whether the access sequence is a valid access sequence. If the expert believes that the access sequence has certain representative significance, that is, the access sequence has certain representative significance, then the access sequence will be included in the above-mentioned task pattern knowledge base.

[0060] 4. Perform visual analysis of complex jobs based on job type.

[0061] like Figure 2 As shown, the present invention uses a user interface to perform multi-scale assessment and visual analysis of complex jobs. The user interface includes four views: outline view, job list view, job space view, and scenario indicator view. The data displayed in these four views are interrelated. The outline view shows the spatiotemporal distribution of all jobs and abnormal trajectories in the job task scenario. The job list view provides detailed information on all jobs within the selected time period. The job space view visualizes the job trajectories. The scenario indicator view shows the spatiotemporal utilization of the job task scenario.

[0062] 1) Outline view.

[0063] To comprehensively analyze complex operational scenarios, the Outline View provides an overview of the spatiotemporal distribution of tasks, allowing users to explore the spatiotemporal distribution of the same task and different tasks. Users can flexibly adjust the time range and spatial location of the analysis as needed, allowing for in-depth analysis of task performance within specific time periods or regions. To optimize operational processes and spatial layouts, the Outline View also displays abnormal spatiotemporal and operational trajectories of agents, allowing users to further explore the source of the anomaly through an interactive interface.

[0064] Specifically, the outline view allows users to visualize and analyze the job distribution of task scenarios using spatial patterns, spatiotemporal patterns, and anomaly analysis modes. The spatial pattern displays job information corresponding to trajectory clusters with the same spatial pattern. Trajectory clusters with the same spatial pattern are obtained by merging spatiotemporal clustering results in the time dimension. In other words, the spatial pattern displays trajectory clusters with the same spatial motion pattern. Each spatial pattern is represented by a thick line. When a user clicks a spatial cluster, the view displays the spatiotemporal clusters contained in that cluster and their proportions through a pattern information panel. These jobs are also sorted to the front of the job list view. This allows users to conduct in-depth analysis of job performance for specific time periods or regions. For example, if a user requires a more granular display of trajectories for a spatial pattern, they can zoom in by scrolling the mouse wheel. The outline view displays each trajectory contained in that spatial pattern. When a user clicks a trajectory, the outline view displays information such as the owner number, type, start time, end time, and duration of the trajectory through a track information panel.

[0065] The spatiotemporal mode displays the spatiotemporal distribution of each job type, that is, jobs with different spatiotemporal characteristics. To prevent visual clutter caused by a large number of spatiotemporal clusters, the outline view defaults to spatial mode. Users can manually switch to spatiotemporal mode.

[0066] The anomaly analysis mode only displays trajectories that do not belong to any cluster or for which the job type cannot be determined.

[0067] The outline view also features a timeline at the bottom. By default, the outline view displays information for all time periods. Users can adjust this timeline to display the distribution of work within the selected time range. Once a time range is selected, the content displayed in the other three views will also change accordingly.

[0068] 2)Job list view.

[0069] The job list view shows users detailed information about all jobs. The job list view has two display modes: type statistics and job details. In the job details mode, the job list view shows users detailed information about each independent job. The job details are determined based on the scenario of complex jobs. For example, for aviation insurance jobs, the job information may include: job number (Id), job type (Type), start time (Start), end time (End), duration (Duration), number of participants (Crew), and number of vehicles occupied (Vehicle), etc. The job types include: refueling (Fueling), ordnance (Ordnance), oxygen and nitrogen filling (Oxy-Nitro), tie-down (Tie-down), etc.

[0070] In Type Statistics mode, the Job List view displays detailed information about a specific type of job. Double-clicking a job will redirect to its details. This view is also linked to the Job Space view: when a user clicks a record in the list, the Job Space view will highlight that track cluster.

[0071] 3) Workspace view.

[0072] The Workspace View transforms continuous track points into intuitive visual elements, providing a microscopic visualization of the work, helping users understand the work process and verify their analysis, enhancing the interpretability of work dynamics. When users select a work of interest in the Workspace View, the Workspace View automatically displays its track distribution, using line thickness to indicate the number of people involved. It also supports zooming in and out. For example, using the mouse wheel to zoom in or out dynamically adjusts the scale of the workspace, allowing for detailed inspection of specific areas. Figure 2 Figure C is the working space view of the flight support operation, with colored solid lines and line segments forming a trajectory; the white text on a blue background is the carrier-based aircraft service position, labeled AH; the red part in the figure is the weapon elevator; the pink and yellow dotted lines represent the carrier-based aircraft elevator, labeled 1-4; and the purple part represents the refueling station.

[0073] 4) Scenario indicator view.

[0074] The scene indicator view is used to display the spatiotemporal usage indicators of each job, thereby presenting the spatiotemporal utilization of the scene. Complex jobs involve multiple important spatial areas. To ensure the efficient use of these spaces, the present invention provides a comprehensive spatiotemporal utilization analysis. Users can view the usage of each area in different time periods through the scene indicator view in the visualization interface, and identify possible inefficient areas or overcrowded areas. Users can choose to view the required spatiotemporal usage indicators by area. In one embodiment, two selection boxes can be set in the scene indicator view, for users to select areas in the complex job scene and the required spatiotemporal usage indicators, respectively. Below these two selection boxes, the changes in the selected indicators of the selected area over time are displayed according to the above selections. The spatiotemporal usage indicators include at least two types of data: job area density, job area out-degree, job area in-degree, job area betweenness centrality, edge weight and edge betweenness centrality.

[0075] Among them, regional density is used to measure regional operation density and reflect the degree of congestion within the operation radius. The regional density calculation formula is:

[0076]

[0077] Among them, U v (t) represents the regional density of region v in the tth time step, N v(t) represents the number of agents in region v in the tth time step, R v Represents the area of ​​region v. Intelligent agents include humans and other mobile devices, such as vehicles, motion robots, etc.

[0078] The region out-degree and region in-degree represent the frequency of direct connections between the region and other regions within a given time period. Based on the access sequence, the transfer events of personnel between regions are extracted, and the frequency of out-edges (leaving) and in-edges (arriving) of each region is counted.

[0079] The betweenness centrality of a region measures the importance of the region as a "bridge" and is calculated as follows:

[0080]

[0081] Where BC(v,t) represents the betweenness centrality of region v in the t-th time step, σ od (v,t) represents the total number of paths from region o to d in the tth time step, σ od (t) represents the number of paths passing through region v in the tth time step.

[0082] Edge weight is used to measure the importance of edges between regions, and the calculation formula is:

[0083] w(u,v)=N uv

[0084] Among them, w(u,v) represents the edge weight between region u and region v, N uv Indicates the number of agents that pass from area u to area v within a set time period.

[0085] Edge betweenness centrality is used to measure the importance of an edge as a "bridge". The calculation formula is:

[0086]

[0087] Where BC(e,t) represents the betweenness centrality of edge e in the t-th time step, e represents the edge between region u and region v, σ sq (e,t) represents the number of paths passing through edge e in the shortest path from region s to region q in the tth time step, σ sq (t) represents the total number of shortest paths from region s to region q in the tth time step.

[0088] Device implementation method

[0089] A visual analysis device for complex jobs of the present invention includes a processor and a user interface. The processor is used to determine the spatiotemporal distribution of all jobs, detailed information of each job, job trajectory, abnormal trajectory and spatiotemporal usage indicators of each job according to the visual analysis method for complex jobs introduced in the above-mentioned method embodiment, and visualize the data through the user interface, so that users can perform visual analysis of complex jobs based on the visualized data.

Claims

1. A visual analysis method for complex tasks, characterized in that: include: Obtain the original trajectory data of the complex operation process and divide the operation scene into grids based on the original trajectory points in the spatial and temporal dimensions; Map the original trajectory points to the corresponding grid to obtain the mapped spatiotemporal trajectory data; The mapped spatiotemporal trajectory data is input into the joint model to obtain trajectory clusters. The joint model includes a clustering model and an optimized neural network model. The optimized neural network model is obtained by first pre-training the neural network model using the minimized reconstruction loss, and then jointly training the pre-trained neural network model using a joint loss that combines the minimized reconstruction loss and the clustering loss. The optimized neural network model is used to extract optimized feature vectors, and the clustering model is used to cluster the extracted optimized feature vectors to obtain trajectory clusters. Determine the job type corresponding to each trajectory cluster and analyze complex jobs based on the job type.

2. The visual analysis method for complex tasks according to claim 1, characterized in that: The clustering loss includes intra-cluster compactness loss, inter-cluster separation loss and balanced regularization loss; wherein, the intra-cluster compactness loss is used to describe the similarity of samples within a trajectory cluster, the inter-cluster separation loss is used to describe the separability between trajectory clusters, and the balanced regularization loss is used to describe the size deviation of trajectory clusters.

3. The visual analysis method for complex tasks according to claim 1 or 2, characterized in that: The process of gridding the work scene according to the original trajectory points is as follows: first, the original trajectory points are clustered from the spatial dimension and the temporal dimension respectively using the density clustering algorithm, and then the work scene is gridded based on the density clustering results. The denser the distribution area of ​​the trajectory points, the smaller the corresponding grid.

4. The visual analysis method for complex tasks according to claim 1, characterized in that: The process of determining the job type corresponding to each trajectory cluster is to determine the access sequence of the trajectory cluster and determine the job type corresponding to the trajectory cluster based on the access sequence; The process of determining the access sequence of a trajectory cluster is to judge whether each trajectory point in the trajectory cluster meets the valid contact event triggering conditions, and obtain the access sequence of the trajectory cluster based on the satisfied trajectory points; among them, the valid contact event triggering conditions are: the distance between the trajectory point and the job-related position is less than the job radius and the continuous stay time exceeds the minimum time threshold required for job execution.

5. The visual analysis method for complex tasks according to claim 4, characterized in that: The process of determining the job type corresponding to a trajectory cluster is to predefine the access sequence template corresponding to each job type, match the access sequence with the access sequence template; if the match is successful, the job type corresponding to the successfully matched access sequence template is used as the job type of the trajectory cluster; If the match is not successful, manual review is used to determine whether it is an abnormal trajectory. If it is not an abnormal trajectory, it is determined whether the access sequence is a valid access sequence. If it is a valid sequence, the job type of the access sequence is marked and added to the access sequence template.

6. The visual analysis method for complex tasks according to claim 1, characterized in that: Minimize the reconstruction loss: L r =-logp(y|x) Among them, L r To minimize the reconstruction loss, y is the original trajectory in the dataset, x is the interference trajectory generated based on y, and p is the probability.

7. The visual analysis method for complex tasks according to claim 5, characterized in that: The analysis of complex jobs based on job types includes displaying the spatiotemporal distribution of all jobs under complex tasks, detailed information on each job, job trajectories, and abnormal trajectories.

8. The visual analysis method for complex tasks according to claim 7, characterized in that: The analysis of complex jobs based on job types also includes determining the spatiotemporal usage indicators of each job. The spatiotemporal usage indicators include at least two types of data: regional density, regional out-degree, regional in-degree, regional betweenness centrality, edge weight, and edge betweenness centrality.

9. A visual analysis device for complex operations, comprising a processor, characterized in that: The processor performs visual analysis of complex operations using the visual analysis method for complex operations as described in any one of claims 1 to 6.

10. The visual analysis device for complex tasks according to claim 9, characterized in that: The device also includes a user interface for visualizing data of complex tasks, It includes displaying the spatiotemporal distribution of all jobs under complex tasks, detailed information of each job, job trajectory, abnormal trajectory and spatiotemporal usage indicators of each job. The spatiotemporal usage indicators include at least two types of data: regional density, regional out-degree, regional in-degree, regional betweenness centrality, edge weight and edge betweenness centrality.

Images