Scenic spot flow prediction method for scenic spot tourist trajectory

Abstract

The invention relates to a scenic spot flow prediction method for scenic spot tourist tracks, and belongs to the technical field of flow prediction, and the method comprises the following steps: S1, laying a video image access network in a scenic spot, including a server and cameras installed at key scenic spots, playing points and entrances and exits of the scenic spot in the scenic spot; s2, performing pedestrian detection and pedestrian re-identification retrieval on the video acquired by the video image access network; detecting and marking pedestrian IDs by cameras arranged at entrances and exits of scenic spots, and obtaining video image information of tourists of the pedestrian IDs appearing in the scenic spots through pedestrian re-identification; s3, constructing a scenic spot trajectory time spectrogram, and obtaining a trajectory route statistical graph based on the adjacency relation directed graph; and S4, constructing a scenic spot trajectory flow diagram time sequence and a scenic spot flow prediction model, and performing prediction and early warning on each scenic spot line and scenic spot flow in the scenic spot.

Description

Technical Field

[0001] The invention belongs to the technical field of flow prediction and relates to a method for predicting flow of scenic spots based on tourist trajectories in scenic spots. Background Art

[0002] With the booming tourism industry, scenic spots hope to conduct statistical analysis on the tourists' visits in scenic spots through various technical means. For example, facial recognition technology is used to analyze the identity of tourists and multi-dimensional features, and the XGBoost model based on multi-feature fusion is used to predict the probability of tourists' transfer from one scenic spot to another, and to warn the real-time number of people; according to the time and duration of tourists' access to the hotspot network of each scenic spot in the scenic spot, the tourist's tour trajectory in the scenic spot and the transfer path between each two scenic spots are inferred; pedestrian re-identification is used to analyze the movement of tourists in the venue, obtain the tourist trajectory data of different areas in the venue, and obtain the appearance time and disappearance time of each tourist's trajectory; by registering and opening a tourist client account in the scenic spot operation management system, authorizing access to the client's location and speed information, and collecting the current location and current moving speed of tourists during the tourist scenic spot travel, and finally calling the scenic spot route database, and combining the scenic spot route database to generate the current action trajectory.

[0003] Although there are some technical means for analyzing the trajectory of tourists in scenic spots, there is a greater need for a technology to predict the flow of tourists in scenic spots based on trajectory analysis, so as to plan routes and control flow in advance. Traditional back propagation neural networks are difficult to capture the complex spatiotemporal relationship between scenic spots. Summary of the invention

[0004] In view of this, the object of the present invention is to provide a method for predicting the flow of scenic spot visitors based on their trajectories.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for predicting the flow of scenic spots based on tourist trajectories in scenic spots comprises the following steps:

[0007] S1: Deploy a video image access network in the scenic area, including servers and cameras installed at key scenic spots, tourist spots and entrances and exits in the scenic area;

[0008] S2: Pedestrian detection and pedestrian re-identification retrieval are performed on the video collected by the video image access network; the camera installed at the entrance and exit of the scenic spot detects and marks the pedestrian ID, and the video image information of tourists with each pedestrian ID appearing in the scenic spot is obtained through pedestrian re-identification;

[0009] S3: Construct a time spectrum diagram of the scenic spot trajectory and obtain a trajectory route statistical diagram based on the adjacency relationship directed graph;

[0010] S4: Construct a time series of scenic spot trajectory flow diagram and a scenic spot flow prediction model to predict and warn the routes and flow of each scenic spot in the scenic area.

[0011] Further, the server sets and records the camera number VideoID, and the scenic spot information (SightID, Sightname) corresponding to the camera number VideoID, and ensures that the video image in the camera can obtain real-time time information; camera number VideoID = {Vidoe1, .., Video k}, k represents the number of cameras in the scenic area; SightID = {Sight1, .., Sight q}, q represents the number of scenic spots in the scenic area; establish Video k and Sight q Relationship, indicating the corresponding relationship between the camera and the scenic spot.

[0012] Further, step S2 specifically includes the following steps:

[0013] S21: using a pedestrian detection model to perform pedestrian detection on the collected video frames, extracting images containing pedestrians, and saving them to a pedestrian image library in the server;

[0014] S22: Find the pedestrian image files of each entrance and exit of the scenic spot from the pedestrian image library, and put them into the entrance and exit image folder; select any pedestrian image from the entrance and exit image folder, mark the pedestrian ID, and perform pedestrian re-identification retrieval on the entrance and exit image folder with this pedestrian ID and the corresponding pedestrian image; move all images with the retrieved pedestrian ID from the entrance and exit image folder into another folder, and mark these moved images with the pedestrian ID and image time; traverse all images and image times in the pedestrian ID folder, find the earliest time record and the latest time record, and judge and record the time when the pedestrian image enters and leaves the scenic spot; repeat the process of step S22 until all pedestrian images in the entrance and exit image folder in a trajectory analysis cycle are marked with pedestrian ID and the time of entering and leaving the scenic spot;

[0015] S23: According to the pedestrian ID marked at the entrance and exit and the time when the pedestrian ID enters and exits the scenic area, the pedestrian images belonging to the time period are filtered out from the pedestrian image library, and pedestrian re-identification retrieval is performed through the pedestrian re-identification retrieval model MSNet; all pedestrian image files corresponding to each pedestrian ID are obtained, and the file name is obtained.

[0016] Further, step S3 specifically includes the following steps:

[0017] S31: adding the pedestrian ID retrieved in step S23 to a marked trajectory sequence in chronological order; in the marked trajectory sequence, recording basic information related to the search file name, the scenic spot bound to the camera, and the video frame corresponding to the search file;

[0018] S32: Track labeling and statistical analysis are performed in two ways:

[0019] The first method is to mark each real track of each tourist according to the time of visit, and count the time period of tourists visiting each scenic spot;

[0020] Construct the time spectrum diagram of tourists' travel trajectories at various scenic spots f(T1+h*Ta,Sight q ), the horizontal axis of the graph is the arrangement value of the time period divided by the time interval Ta during the business hours of the scenic spot (T1, T1+Ta,…, T1+h*Ta), T1 is the initial business hours of the scenic spot; the vertical axis of the graph is the arrangement value of each scenic spot (Sight1,.., Sight q ); Taking a time interval Ta as a unit, count the Time in the sequence information of all tourists’ Tracks in each time interval it The distribution of tourists falling on all attractions in this time interval;

[0021] The second method: According to the trajectory route, the routes of tourists in each time interval are counted: all the paths of the scenic spots are drawn into a directed graph. The edges on the directed graph represent the connected adjacency relationship from one scenic spot to another. The weight is marked on the edge. The weight is the number of adjacencies in the trajectory of all tourist attractions in the time interval, that is, the number of tourists moving from one scenic spot to another.

[0022] Furthermore, the second method specifically includes:

[0023] Construct a directed graph G(V, E, w), where V represents the scenic spots in the graph, and each scenic spot (Entrance, Sight1, .., Sight q , Exit); E represents the edge in the graph, which is a directed edge that can connect various scenic spots; w is the weight of each directed edge, which represents the number of tourists on the directed edge; the weights between each vertex and edge on the directed graph G are counted to show the tourist travel routes;

[0024] Track each visitor within the time interval Ta i Process the adjacency relationship; traverse the pedestrian person i Track sequence i , in Track i Integrate the video image information, press Time it The order of the same SightID in the sequenceit Image it Ignore, different SightID it Image it Save to Track1 i Track 1 i =(Image i1 ,Image i2 ,…,Image is ), Image is =(SightID is ,Time is ) stores the track of the pedestrian ID appearing at the scenic spot in chronological order.

[0025] Further, step S4 specifically includes the following steps:

[0026] S41: construct a time series of scenic spot trajectory flow diagram, specifically including: taking the time spectrum diagram of each scenic spot playing trajectory f(T1+h*Ta,Sight q ) and the directed graph G of tourist routes t Construct a time series. The time interval is represented by the time interval in the trajectory time spectrum graph. The directed graph G on each time interval is statistically constructed according to the edge weights of the adjacency relationship of all tourist trajectories in the time interval. T The edge features include the edge weight statistics w and the passable state state of the road section corresponding to the edge. When state = 1, it means that the road section is passable, and when state = 0, it means that the road section is not passable. The node features of the directed graph in the time period T include the time spectrum graph f(T, Sight q ) Tourist flow statistics and tourist attraction-related factors of all attractions during that time;

[0027] S42: Construct a scenic spot flow prediction model, and use the scenic spot trajectory flow map time series as the input of the prediction model, specifically including:

[0028] S421: Message passing: Aggregate node and edge information. At each time step T, the features of node i and neighbor node j are and The characteristics of edge (i,j) are The memory state of the edge is The message aggregation process of the multi-head attention mechanism is:

[0029] Calculate the attention weights, for each attention head k:

[0030]

[0031] in: is the attention weight between node i and its neighbor node j on the kth head; W k is the weight matrix of the kth head; a k ′ is the transpose of the attention parameter vector of the kth head; || represents the vector concatenation operation; LeakyReLU is the LeakyReLU activation function;

[0032] Compute the weighted message, for each attention head k:

[0033]

[0034] in, is a learnable weight matrix;

[0035] Aggregate the messages of all attention heads:

[0036]

[0037] Where K is the number of attention heads;

[0038] S422: Edge memory update: Set the new edge feature received each time Update the memory of the corresponding edge Where, φ is the edge memory update function;

[0039] S423: Node status update: Use edge memory to update the node status;

[0040] Use the edge memory state and neighbor node information to update the node state. For node i, its new state can be expressed as follows:

[0041]

[0042] in, is the node status update function;

[0043] S424: Predict edge weights, i.e., the tourist flow from one attraction to another attraction (i→j):

[0044]

[0045] Predict the tourist flow of node characteristics:

[0046]

[0047] y i is the feature value prediction of node i; W y is the weight matrix of node feature value prediction; b y is the bias term for node feature value prediction; σ is the activation function; is the node and edge state after connection; is the aggregate edge memory of node i; is the node state of the integrated node state and aggregated edge memory; w ij is the weighted prediction; W w is the weight matrix of edge weight prediction; b w is the bias term for edge weight prediction;

[0048] S43: Setting a flow threshold of the scenic spot, and issuing a flow warning when the predicted flow of the scenic spot exceeds the threshold.

[0049] Further, in step S41, the constructed scenic spot trajectory flow diagram time series is SG = (G1, G2, ..., G j ,…G T ), G j is a directed graph of tourist attraction trajectory statistics at the jth time interval, j = 1, ..., T, T represents the time interval information of the time series, G j (V j , A j , w j ), where V j Represents the scenic spots and their features in the graph. The scenic spot set of the scenic area is {Entrance, Sight1, .., Sight q , Exit}, represented by {V1,V2,…,V k ,…,V q},V j The node features of each scenic spot combination are represented by vector {V j1 ,V j2 ,…,V jk ,…,V jq}, where V jk V represents the characteristic information of the kth scenic spot in the jth time interval; jk Including the time spectrum information of the scenic spot f(j,Sight k ) and other factors related to scenic spot visits; E j Represents the edge set and edge features in the graph. The set of edges between scenic spots is {(Sight1→Sight1),…,(Sight k →Sight q ),…}, the set of edges is directional, representing the paths between scenic spots, and is represented by an adjacency matrix; the edge feature is composed of the weight of the edge in the time interval j and the passable state state of the road section between the two scenic spots represented by the edge, which is a two-dimensional vector.

[0050] Furthermore, in step S422, the memory update function of the edge is GRU or LSTM:

[0051] or

[0052] is the memory cell state of LSTM edge (i,j), is the edge memory after edge (i,j) is updated.

[0053] Further, in step S423, the node state update function is GRU or LSTM:

[0054] or

[0055] is the memory cell state of LSTM node i, is the updated state of node i.

[0056] Further, step S43 specifically includes the following steps:

[0057] S431: Constructing a scenic spot threshold vector and a scenic spot path threshold matrix: For each scenic spot and scenic spot path, a tourist flow threshold is set based on the capacity, safety factors and historical data of the scenic spot or scenic spot path; for each scenic spot V i , set the corresponding threshold Form a threshold vector of all scenic spots Where q is the total number of nodes; for each path connecting the scenic spots, a path flow threshold is set; each edge E is set i,j The weight threshold is Form a threshold matrix Θ E ,in Indicates the view from the scenic spot V i To the attractions j Path flow threshold;

[0058] S432: Traffic prediction and comparison: Use the scenic spot traffic prediction model to predict the traffic of scenic spots and paths in the scenic area, and obtain the scenic spot prediction value y i and the predicted value of the scenic spot path w ij ; Compare the obtained prediction value with the defined prediction value: For each scenic spot V i ,if Then the traffic of the scenic spot exceeds the threshold; for each path E i,j ,if Then the flow of the path exceeds the threshold;

[0059] S433: Node warning: If the flow of a certain scenic spot exceeds the threshold Trigger the traffic warning of the scenic spot and notify the scenic area management staff to take flow limiting and diversion measures; Path warning: If the flow of a certain path exceeds the threshold Trigger an early warning for the path, close the path or guide tourists to choose other paths.

[0060] The beneficial effects of the present invention are as follows: the present invention obtains the video image information of tourists appearing in the cameras of various scenic spots in the scenic area through pedestrian detection and pedestrian re-identification, constructs a tourist trajectory marking and analysis module, establishes a time spectrum diagram of tourists' playing trajectories in various scenic spots, and counts the tourist flow of various scenic spots in the scenic area within a certain period of time in the form of a graph structure. Based on the time spectrum diagram of playing trajectories and the tourist flow diagram, a graph time series and period prediction model is established to predict and warn the routes and flow of various scenic spots in the scenic area.

[0061] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0063] Figure 1 It is a flow chart of the scenic spot flow prediction method in the present invention;

[0064] Figure 2 It is a time spectrum diagram of the playing trajectory of each scenic spot of the present invention;

[0065] Figure 3 It is a directed graph of tourist travel routes of the present invention;

[0066] Figure 4 It is the overall flow chart of the prediction process in the present invention;

[0067] Figure 5 This is a structural diagram of the prediction network in the present invention. DETAILED DESCRIPTION

[0068] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0069] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0070] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0071] like Figure 1-5 As shown, the present invention provides a method for predicting scenic spot flow based on the tourist trajectory of the scenic spot, which identifies and analyzes the sightseeing trajectory of the tourists in the scenic spot, including the following steps:

[0072] S1: Video access network: Build a video image access network in the scenic spot. Deploy a video access network in the scenic spot to ensure that each access camera can clearly capture tourists in the key scenic spots or tourist spots in the scenic spot, including the entrances and exits of the scenic spot. Set and record the camera number VideoID, as well as the scenic spot information (SightID, Sightname) corresponding to the camera number VideoID, and ensure that the video image in the camera can obtain real-time time information. Camera number VideoID = {Vidoe1, .., Video k}, k represents the number of cameras in the scenic area. SightID = {Sight1, .., Sight q}, q represents the number of scenic spots in the scenic area. k and Sight q Relationship, indicating the corresponding relationship between the camera and the scenic spot.

[0073] S2: Pedestrian detection and re-identification module: Pedestrian detection and pedestrian re-identification retrieval are performed on the video accessed by the scenic spot. The cameras set up at the entrance and exit of the scenic spot detect and mark the pedestrian ID, and the video image information of tourists with each pedestrian ID appearing in the scenic spot is obtained through pedestrian re-identification. Specifically including:

[0074] S21: In the camera access network deployed in the scenic area, set the video frame sampling interval for image detection: one frame every 30 seconds. Use pedestrian detection models, such as the YOLO series or the SlowFast model, to detect pedestrians in all video frame images, frame out and save the pedestrian information, and the saved detected pedestrian image file information includes: camera number VideoID, time Time, video segment number SegID, video frame number FrameID, and pedestrian number No. in the frame. The video segment number SegID is divided into time periods by day. For example, within the 10 hours from 8:00 to 18:00, it is divided into half an hour, and SegID is an integer from 1 to 20; the video frame number FrameID refers to the sequence number of the video frame in a certain video segment, corresponding to the video frame Frame. The detected pedestrian image file is stored in the pedestrian image library on the server.

[0075] The video frame (Vidoe1, 6, 2024-06-18 09:25:30, 3254, 3) is represented as (01 06202406180925303254 03), where 01 represents the camera number, 06 represents the video segment number of the video frame, 20240618092530 represents the time of the video frame, 3254 represents the video frame number, and 03 represents the pedestrian number in the frame.

[0076] S22: From the pedestrian image library, find the detected pedestrian image files of each entrance and exit of the scenic spot, and put them into the entrance and exit image folder. Adopt the random crawling principle, select pedestrian images at random, mark the pedestrian ID, and use this pedestrian ID and pedestrian image to perform pedestrian re-identification retrieval on the entrance and exit image folder, move all images with the retrieved pedestrian ID from the entrance and exit image folder into another folder, and mark these moved images with the pedestrian ID and image time. Traverse all images and image times in the pedestrian ID folder, find the earliest time record and the latest time record, and judge and record the time when the pedestrian image enters and leaves the scenic area. This random crawling, identification and movement process ends until all pedestrian images in the entrance and exit image folder in a trajectory analysis cycle are marked with pedestrian IDs and the time of entering and leaving the scenic area. Here, the set of pedestrian IDs detected and marked in the same week is selected as IDset = {person1,…,person i ,…,person n}, person i is the ith pedestrian ID,

[0077] S23: According to the pedestrian ID marked at the entrance and exit and the time when the pedestrian ID enters and exits the scenic area, the pedestrian images belonging to the time period are screened from the pedestrian image library to perform pedestrian re-identification retrieval. The pedestrian re-identification retrieval model can use MSNet. It is a lightweight multi-scale network that combines multi-scale feature fusion and attention mechanism. By efficiently extracting key features from pedestrian images, it can significantly improve the performance of re-identification. Get all pedestrian image files corresponding to each pedestrian ID and get the file name. From the camera VideoID, scenic spot SightID, video frame time Time, video segment SegID and video frame number FrameID in the file name, extract the video frame Frame containing the pedestrian ID from the camera.

[0078] S3: Tourist trajectory marking and analysis module: constructs a time spectrum diagram of scenic spot trajectories and a trajectory route statistics diagram based on the adjacency relationship directed graph. Specifically includes:

[0079] S31: Add the pedestrian ID retrieved in S23 to the marked trajectory sequence in chronological order. The sequence arrangement is determined by the time and video frame number of the video frame. In the sequence, the basic information involved in the retrieval file name, the scenic spot bound to the camera, and the video frame corresponding to the retrieval file are recorded. i The retrieved trajectory sequence is represented as Track i =(Image i1 ,Image i2 ,…,Image it ), where Image it Represents the information about the retrieved file at the tth time in the sequence. Image it =(VideoID it ,SightID it ,Time it ,SegID it ,FrameID it ,Frame it ), respectively representing the i-th tourist person i The camera number, scenic spot number, time and video segment and video frame number and video frame of the scenic spot in the trajectory sequence at time t.

[0080] S32: Track labeling and statistical analysis are performed in two ways:

[0081] 1. Mark each real track of each tourist according to the time of visit, and count the time period of tourists visiting each scenic spot.

[0082] Construct the time spectrum diagram of tourists' travel trajectories at various scenic spots f(T1+h*Ta,Sight q ), the horizontal axis of the graph is the time period arrangement value divided by the time interval Ta during the business hours of the scenic spot (T1, T1+Ta,…, T1+h*Ta), T1 is the initial business hours of the scenic spot, for example, starting from 8:00 in the morning, a time coordinate point is marked every 10 minutes, T1=8:00, Ta=10; the vertical axis of the graph is the arrangement value of each scenic spot (Sight1,..,Sight q ).

[0083] Take a time interval Ta as a unit, and count the Time in the sequence information of all tourists' Tracks in each time interval it The distribution of tourists falling on all attractions in this time interval.

[0084] Taking the T1+Ta time period as an example, the algorithm is as follows:

[0085]

[0086] This statistical analysis method can obtain the number and distribution of tourists at each scenic spot in each time period of the scenic area, which is a reflection of the tourist volume of the scenic spot in each time period of the scenic area.

[0087] 2. Count the routes of tourists in each time interval according to the trajectory route: Draw all the paths of the scenic spots into a directed graph. The edges on the directed graph represent the connected adjacency relationship from one scenic spot to another. Mark the weight on the edge, which is the number of adjacencies in all tourist attraction trajectories in the time interval, that is, the number of tourists moving from one attraction to another.

[0088] Construct a directed graph G(V, E, w), where V represents the scenic spots in the graph, and each scenic spot (Entrance, Sight1, .., Sight q , Exit), E represents the edge in the graph, which is a directed edge that can connect the scenic spots, such as (Sight k →Sight q ), w is the weight of each directed edge, such as w(Sight k →Sight q ), denoted as w kq , is the scenic spot Sight k To Sight q The weights of the vertices and edges on the directed graph G are counted to show the tourist routes.

[0089] Track each visitor within the time interval Ta i Perform adjacency processing. Traverse the pedestrian person iTrack sequence i , in Track i Integrate the video image information, press Time it The order of the same SightID in the sequence it Image it Ignore, different SightID it Image it Save to Track1 i Track 1 i =(Image i1 ,Image i2 ,…,Image is ), Image is =(SightID is ,Time is ) only stores the trajectory of the pedestrian ID appearing at the scenic spot in chronological order. The adjacency relationship of the trajectories of all tourists within the time interval Ta is analyzed, and the algorithm is as follows:

[0090]

[0091] This statistical analysis method can obtain a statistical chart of the travel routes of all tourists in the scenic area within the time interval Ta.

[0092] S4: Scenic spot flow prediction module: Construct the time series of scenic spot trajectory flow diagram and scenic spot flow prediction model to predict and warn the routes and flow of each scenic spot in the scenic area. Specifically include:

[0093] S41: Constructing a time series of scenic spot trajectory flow graphs

[0094] The time spectrum of the tourist trajectories of each scenic spot f(T1+h*Ta,Sight q ) and the directed graph G of tourist routes t , constructing the time series, the time interval can be represented by the time interval in the trajectory time spectrum graph, and the directed graph G on each time interval is statistically constructed according to the edge weights of the adjacency relationship of all tourist trajectories in the time interval. T In the edge features, in addition to the edge weight statistics w, there is also the passable state state of the road section corresponding to the edge. When state = 1, it means that it is passable; when state = 0, it means that the road section is not passable due to accidents, maintenance, traffic control, etc. The node features of the directed graph in the T time period include the time spectrum graph f(T,Sight q) in addition to the tourist flow statistics of all attractions during that time, there are also factors related to sightseeing, such as landscape category, distance and means of transportation, weather, special activities and ratings, attraction theme, attraction facilities or services, photography spots or check-in spots and other points of interest.

[0095] Assume that the constructed scenic spot trajectory flow diagram time series is SG = (G1, G2, ..., G j ,…G T ), G j is a directed graph of tourist attraction trajectory statistics at the jth time interval, j = 1, ..., T, T represents the time interval information of the time series, G j (V j , A j , w j ), where V j Represents the scenic spots and their features in the graph. The scenic spot set of the scenic area is {Entrance, Sight1, .., Sight q , Exit}, represented by {V1,V2,…,V k ,…,V q},V j The node features of each scenic spot combination are represented by vector {V j1 ,V j2 ,…,V jk ,…,V jq}, where V jk V represents the characteristic information of the kth scenic spot in the jth time interval. jk Including the time spectrum information of the scenic spot f(j,Sight k ) and other factors related to scenic spot visits. j Represents the edge set and edge features in the graph. The set of edges between scenic spots is {(Sight1→Sight1),…,(Sight k →Sight q ),…}, the set of edges is directional, representing the paths between scenic spots, and is represented by an adjacency matrix. The edge feature consists of the weight of the edge in the time interval j and the passable state of the road section between the two scenic spots represented by the edge, which is a two-dimensional vector. For example: edge E j (Sight k →Sight q ) represents an edge from scenic spot k to scenic spot q within time interval j. The feature of the edge is expressed as E under the condition of being passable. j,(k,q) =[w kq ,1].

[0096] S42: Building a tourist attraction traffic prediction model

[0097] The scenic spot traffic prediction model is constructed based on the scenic spot trajectory traffic graph time series SG:

[0098] S421: Message passing: Aggregate node and edge information. At each time step T, for the convenience of formula expression, assume that the characteristics of node i and neighbor node j are and The characteristics of edge (i,j) are The memory state of the edge is The message aggregation process of the multi-head attention mechanism is:

[0099] Calculate the attention weights and set multiple attention heads k at each message passing step. For each attention head k:

[0100]

[0101] in: is the attention weight between node i and its neighbor node j on the kth head. k is the weight matrix of the kth head. k ′ is the transpose of the attention parameter vector of the kth head. || represents the vector concatenation operation. LeakyReLU is the LeakyReLU activation function.

[0102] Compute the weighted message, for each attention head k:

[0103] in, is a learnable weight matrix.

[0104] Aggregate the messages of all attention heads:

[0105] Where K is the number of attention heads.

[0106] S422: Edge memory update: Set the new edge feature received each time Update the memory of the corresponding edge where φ is the edge memory update function.

[0107] Optionally, the edge memory update function can be GRU or LSTM:

[0108] or

[0109] is the memory cell state of LSTM edge (i,j), is the edge memory after edge (i,j) is updated.

[0110] S423: Node status update: Use edge memory to update the node status.

[0111] Use the edge memory state and neighbor node information to update the node state. For node i, its new state can be expressed as follows:

[0112]

[0113] in, It is the node status update function.

[0114] Optionally, the node state update function can be GRU or LSTM:

[0115] or

[0116] is the memory cell state of LSTM node i, is the updated state of node i.

[0117] S424: Predict edge weight (tourist flow from one attraction to another attraction (i→j)):

[0118]

[0119] Predict the tourist flow of node features (attraction i):

[0120]

[0121] y i is the predicted eigenvalue of node i. y is the weight matrix of node feature value prediction. y is the bias term for node feature value prediction. σ is the activation function. are the node and edge states after connection. is the aggregate edge memory of node i. is the node state that integrates the node state and aggregated edge memory. ij is the weighted prediction. w is the weight matrix of edge weight prediction. b w is the bias term for edge weight prediction.

[0122] S43: Setting a flow threshold of the scenic spot, and issuing a flow warning when the predicted flow of the scenic spot exceeds the threshold. Step S43 specifically includes the following steps:

[0123] S431: Constructing a scenic spot threshold vector and a scenic spot path threshold matrix: For each scenic spot and scenic spot path, a tourist flow threshold is set based on the capacity, safety factors and historical data of the scenic spot or scenic spot path; for each scenic spot V i , set the corresponding threshold Form a threshold vector of all scenic spots Where q is the total number of nodes; for each path connecting the scenic spots, a path flow threshold is set; each edge E is set i,j The weight threshold is Form a threshold matrix Θ E ,in Indicates the view from the scenic spot V i To the attractions j Path flow threshold;

[0124] S432: Traffic prediction and comparison: Use the scenic spot traffic prediction model to predict the traffic of scenic spots and paths in the scenic area, and obtain the scenic spot prediction value y i and the predicted value of the scenic spot path w ij ; Compare the obtained prediction value with the defined prediction value: For each scenic spot V i ,if Then the traffic of the scenic spot exceeds the threshold; for each path E i,j ,if Then the flow of the path exceeds the threshold;

[0125] S433: Node warning: If the flow of a certain scenic spot exceeds the threshold Trigger the traffic warning of the scenic spot and notify the scenic area management staff to take flow limiting and diversion measures; Path warning: If the flow of a certain path exceeds the threshold Trigger an early warning for the path, close the path or guide tourists to choose other paths.

[0126] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0127] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0128] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0129] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0130] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0131] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0132] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0133] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0134] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0135] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0136] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for predicting the flow of scenic spot tourists based on their trajectories, characterized by: The following steps are involved: S1: Deploy a video image access network in the scenic area, including servers and cameras installed at key scenic spots, tourist spots and entrances and exits in the scenic area; S2: Pedestrian detection and pedestrian re-identification retrieval are performed on the videos collected by the video image access network; The cameras installed at the entrance and exit of the scenic spot detect and mark the pedestrian IDs, and obtain the video image information of tourists with each pedestrian ID appearing at the scenic spot through pedestrian re-identification; S3: Construct a time spectrum diagram of the scenic spot trajectory and obtain a trajectory route statistical diagram based on the adjacency relationship directed graph; S4: Construct a time series of scenic spot trajectory flow diagram and a scenic spot flow prediction model to predict and warn the routes and flow of each scenic spot in the scenic area.

2. The method for predicting scenic spot flow based on tourist trajectories according to claim 1, characterized in that: The server sets and records the camera number VideoID, and the scenic spot information (SightID, Sightname) corresponding to the camera number VideoID, and ensures that the video image in the camera can obtain real-time time information; camera number VideoID = {Vidoe1, .., Video k }, k represents the number of cameras in the scenic area; SightID = {Sight1, .., Sight q }, q represents the number of scenic spots in the scenic area; establish Video k and Sight q Relationship, indicating the corresponding relationship between the camera and the scenic spot.

3. The method for predicting scenic spot flow based on tourist trajectories according to claim 1 is characterized by: Step S2 specifically includes the following steps: S21: using a pedestrian detection model to perform pedestrian detection on the collected video frames, extracting images containing pedestrians, and saving them to a pedestrian image library in the server; S22: Find the pedestrian image files of each entrance and exit of the scenic spot from the pedestrian image library, and put them into the entrance and exit image folder; select any pedestrian image from the entrance and exit image folder, mark the pedestrian ID, and perform pedestrian re-identification retrieval on the entrance and exit image folder with this pedestrian ID and the corresponding pedestrian image; move all images with the retrieved pedestrian ID from the entrance and exit image folder into another folder, and mark these moved images with the pedestrian ID and image time; traverse all images and image times in the pedestrian ID folder, find the earliest time record and the latest time record, and judge and record the time when the pedestrian image enters and leaves the scenic spot; repeat the process of step S22 until all pedestrian images in the entrance and exit image folder in a trajectory analysis cycle are marked with pedestrian ID and the time of entering and leaving the scenic spot; S23: According to the pedestrian ID marked at the entrance and exit and the time when the pedestrian ID enters and exits the scenic area, the pedestrian images belonging to the time period are filtered out from the pedestrian image library, and pedestrian re-identification retrieval is performed through the pedestrian re-identification retrieval model MSNet; all pedestrian image files corresponding to each pedestrian ID are obtained, and the file name is obtained.

4. The method for predicting scenic spot flow based on tourist trajectories according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31: adding the pedestrian ID retrieved in step S23 to a marked trajectory sequence in chronological order; in the marked trajectory sequence, recording basic information related to the search file name, the scenic spot bound to the camera, and the video frame corresponding to the search file; S32: Track labeling and statistical analysis are performed in two ways: The first method is to mark each real track of each tourist according to the time of visit, and count the time period of tourists visiting each scenic spot; Construct the time spectrum diagram of tourists' travel trajectories at various scenic spots f(T1+h*Ta,Sight q ), the horizontal axis of the graph is the arrangement value of the time period divided by the time interval Ta during the business hours of the scenic spot (T1, T1+Ta,…, T1+h*Ta), T1 is the initial business hours of the scenic spot; the vertical axis of the graph is the arrangement value of each scenic spot (Sight1,.., Sight q ); Taking a time interval Ta as a unit, count the Time in the sequence information of all tourists’ Tracks in each time interval it The distribution of tourists falling on all attractions in this time interval; The second method: According to the trajectory route, the routes of tourists in each time interval are counted: all the paths of the scenic spots are drawn into a directed graph. The edges on the directed graph represent the connected adjacency relationship from one scenic spot to another. The weight is marked on the edge. The weight is the number of adjacencies in the trajectory of all tourist attractions in the time interval, that is, the number of tourists moving from one scenic spot to another.

5. The method for predicting scenic spot flow based on tourist trajectories according to claim 4 is characterized by: The second method specifically includes: Construct a directed graph G(V, E, w), where V represents the scenic spots in the graph, and each scenic spot (Entrance, Sight1, .., Sight q , Exit); E represents the edge in the graph, which is a directed edge that can connect various scenic spots; w is the weight of each directed edge, which represents the number of tourists on the directed edge; the weights between each vertex and edge on the directed graph G are counted to show the tourist travel routes; Track each visitor within the time interval Ta i Process the adjacency relationship; traverse the pedestrian person i Track sequence i , in Track i Integrate the video image information, press Time it The order of the same SightID in the sequence it Image it Ignore, different SightID it Image it Save to Track1 i Track 1 i =(Image i1 ,Image i2 ,…,Image is ), Image is =(SightID is ,Time is ) stores the track of the pedestrian ID appearing at the scenic spot in chronological order.

6. The method for predicting scenic spot flow based on tourist trajectories according to claim 1, characterized in that: Step S4 The specific steps include: S41: construct a time series of scenic spot trajectory flow diagram, specifically including: taking the time spectrum diagram of each scenic spot playing trajectory f(T1+h*Ta,Sight q ) and the directed graph G of tourist routes t Construct a time series. The time interval is represented by the time interval in the trajectory time spectrum graph. The directed graph G on each time interval is statistically constructed according to the edge weights of the adjacency relationship of all tourist trajectories in the time interval. T The edge features include the edge weight statistics w and the passable state state of the road section corresponding to the edge. When state = 1, it means that the road section is passable, and when state = 0, it means that the road section is not passable. The node features of the directed graph in the time period T include the time spectrum graph f(T, Sight q ) Tourist flow statistics and tourist attraction-related factors of all attractions during that time; S42: Construct a scenic spot flow prediction model, and use the scenic spot trajectory flow map time series as the input of the prediction model, specifically including: S421: Message passing: Aggregate node and edge information. At each time step T, the features of node i and neighbor node j are and The characteristics of edge (i,j) are The memory state of the edge is The message aggregation process of the multi-head attention mechanism is: Calculate the attention weights, for each attention head k: in: is the attention weight between node i and its neighbor node j on the kth head; W k is the weight matrix of the kth head; a k ′ is the transpose of the attention parameter vector of the kth head; || represents the vector concatenation operation; LeakyReLU is the LeakyReLU activation function; Compute the weighted message, for each attention head k: in, is a learnable weight matrix; Aggregate the messages of all attention heads: Where K is the number of attention heads; S422: Edge memory update: Set the new edge feature received each time Update the memory of the corresponding edge Where, φ is the edge memory update function; S423: Node status update: Use edge memory to update the node status; Use the edge memory state and neighbor node information to update the node state. For node i, its new state can be expressed as follows: in, is the node status update function; S424: Predict edge weights, i.e., the tourist flow from one attraction to another attraction (i→j): Predict the tourist flow of node characteristics: y i is the feature value prediction of node i; W y is the weight matrix of node feature value prediction; b y is the bias term for node feature value prediction; σ is the activation function; is the node and edge state after connection; is the aggregate edge memory of node i; is the node state that integrates the node state and aggregated edge memory; w ij is the weighted prediction; W w is the weight matrix of edge weight prediction; b w is the bias term for edge weight prediction; S43: Setting a flow threshold of the scenic spot, and issuing a flow warning when the predicted flow of the scenic spot exceeds the threshold.

7. The method for predicting the flow of scenic spot tourists according to claim 6 is characterized by: In step S41, the constructed scenic spot trajectory flow diagram time series is SG = (G1, G2, ..., G j ,…G T ), G j is a directed graph of tourist attraction trajectory statistics at the jth time interval, j = 1, ..., T, T represents the time interval information of the time series, G j (V j , A j , w j ), where V j Represents the scenic spots and their features in the graph. The scenic spot set of the scenic area is {Entrance, Sight1, .., Sight q , Exit}, represented by {V1,V2,…,V k ,…,V q },V j The node features of each scenic spot combination are represented by vector {V j1 ,V j2 ,…,V jk ,…,V jq }, where V jk V represents the characteristic information of the kth scenic spot in the jth time interval; jk Including the time spectrum information of the scenic spot f(j,Sight k ) and other factors related to scenic spot visits; E j Represents the edge set and edge features in the graph. The set of edges between scenic spots is {(Sight1→Sight1),…,(Sight k →Sight q ),…}, the set of edges is directional, representing the paths between scenic spots, and is represented by an adjacency matrix; the edge feature is composed of the weight of the edge in the time interval j and the passable state state of the road section between the two scenic spots represented by the edge, which is a two-dimensional vector.

8. The method for predicting scenic spot flow based on tourist trajectories according to claim 6, characterized in that: In step S422, the memory update function of the edge is GRU or LSTM: or is the memory cell state of LSTM edge (i,j), is the edge memory after edge (i,j) is updated.

9. The method for predicting scenic spot flow based on tourist trajectories according to claim 6, characterized in that: In step S423, the node state update function is GRU or LSTM: or is the memory cell state of LSTM node i, is the updated state of node i.

10. The method for predicting scenic spot flow based on tourist trajectories according to claim 6, characterized in that: Step S43 specifically includes the following steps: S431: Constructing a scenic spot threshold vector and a scenic spot path threshold matrix: For each scenic spot and scenic spot path, a tourist flow threshold is set based on the capacity, safety factors and historical data of the scenic spot or scenic spot path; for each scenic spot V i , set the corresponding threshold Form a threshold vector of all scenic spots Where q is the total number of nodes; for each path connecting the scenic spots, a path flow threshold is set; each edge E is set i,j The weight threshold is Form a threshold matrix Θ E ,in Indicates the view from the scenic spot V i To the attractions j Path flow threshold; S432: Traffic prediction and comparison: Use the scenic spot traffic prediction model to predict the traffic of scenic spots and paths in the scenic area, and obtain the scenic spot prediction value y i and the predicted value of the scenic spot path w ij ; Compare the obtained prediction value with the defined prediction value: For each scenic spot V i ,if Then the traffic of the scenic spot exceeds the threshold; for each path E i,j ,if Then the flow of the path exceeds the threshold; S433: Node warning: If the flow of a certain scenic spot exceeds the threshold Trigger the traffic warning of the scenic spot and notify the scenic area management staff to take flow limiting and diversion measures; Path warning: If the flow of a certain path exceeds the threshold Trigger an early warning for the path, close the path or guide tourists to choose other paths.

Images