Pedestrian trajectory sketching method and system supporting cross-domain recognition in edge intelligence environment

Abstract

The application discloses a kind of edge intelligent environment under the support of cross-domain identification pedestrian trajectory delineation method and system, including the image containing person obtained, utilize target detection model to extract person feature;The person feature is clustered according to preset feature clustering formula, and the corresponding feature matrix is generated;When it is confirmed as the same person based on the feature matrix and the person feature information base established in advance, the feature matrix and edge server device identifier are uploaded to cloud server together, so that cloud server determines the specific information of person based on the person information database matched with the person feature information base, and based on the specific information of person and all edge server device identifiers that person passes through, pedestrian trajectory is drawn.The application can obtain the complete behavior trajectory of person, and is delineated on map.

Description

Technical Field

[0001] This invention belongs to the field of pedestrian trajectory characterization technology, specifically relating to a pedestrian trajectory characterization method, device and system that supports cross-domain recognition in an edge intelligence environment. Background Technology

[0002] With the increasing demand for public safety, the number of surveillance cameras has also increased significantly. Currently, Person Re-Identification (Person ReID) aims to solve the problem of quickly and accurately matching and identifying the same pedestrian captured by different cameras. Its core task is to determine whether two different images belong to the same pedestrian. Person Re-Identification has a wide range of applications, including video surveillance, intelligent security, intelligent transportation, pedestrian flow statistics, and consumer behavior analysis. For example, in the field of video surveillance, Person Re-Identification can help security personnel quickly and accurately identify and track suspects, improving the efficiency and accuracy of video surveillance; in the field of intelligent transportation, Person Re-Identification can help traffic management departments monitor pedestrian traffic, improve traffic flow control capabilities, and provide data support for urban traffic planning. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a method and system for pedestrian trajectory characterization that supports cross-domain recognition in an edge intelligence environment. This method can acquire complete behavioral trajectories of individuals and characterize them on a map.

[0004] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0005] In a first aspect, the present invention provides a method for characterizing pedestrian trajectories that supports cross-domain recognition in an edge intelligence environment, applied to an edge server device, comprising:

[0006] For the acquired images containing people, the features of the people are extracted using an object detection model;

[0007] The character features are clustered according to a preset feature clustering formula to generate a corresponding feature matrix;

[0008] If the same person is identified based on the feature matrix and the pre-established database of personal feature information, the feature matrix and the edge server device identifier are uploaded to the cloud server. The cloud server then determines the person's specific information based on the database of personal information that matches the database of personal feature information, and draws the person's trajectory based on the person's specific information and all the edge server device identifiers the person has passed through.

[0009] Optionally, the target detection model is obtained through the following method:

[0010] Set the backbone feature extraction network in the YOLOv5 object detection framework to wide-resNet;

[0011] When using object detection models to extract features from people, wide-resNet extracts three layers of feature maps, selecting the second and third layers as the features of the person.

[0012] Optionally, the method for constructing the personal feature information database includes:

[0013] For different individuals, pre-defined feature clustering and fusion steps are performed to obtain the corresponding clustering and fusion feature matrix;

[0014] Based on the clustering and fusion feature matrices corresponding to different individuals, a database of individual feature information is constructed.

[0015] The preset feature clustering fusion step includes:

[0016] For images of the same person in different dimensions, the backbone feature extraction network of the object detection model is used to extract the corresponding feature maps, forming Q... all Q all ={Q1,Q2,Q3,Q4,...}, where Q1,Q2,Q3,Q4,... represent feature maps in different dimensions;

[0017] The feature maps under the aforementioned multi-dimensional structure are clustered according to a preset feature clustering formula to obtain the corresponding initial clustering fusion feature matrix U; the mathematical expression of the initial clustering feature matrix U is:

[0018]

[0019] in, The characteristic matrix is ​​obtained by analyzing the characteristic map Q. d Feature compression is performed, followed by local feature aggregation using adaptive average feature pooling; (a, b) is the feature matrix. The eigenvalues ​​in f agg For the adaptive average feature pooling function, T all This represents the feature matrix of all person information processed by the pedestrian re-identification model T, with values ​​ranging from 1 to n. i,j Let represent the feature matrix of the i-th layer and j-th dimension after passing through the pedestrian re-identification model T;

[0020] The initial clustering fusion feature matrix U is compressed using a random forest greedy algorithm to obtain the clustering fusion feature matrix. The expression is:

[0021]

[0022] Where m and n are any two eigenvalues ​​in U.

[0023] Optionally, each edge server device stores a camera device table E to store camera device information. The storage format of the camera device table E is as follows:

[0024] E = {Service:P id ,Camera:{α1(x1,y1,z1),α2(x2,y2,z2),α3(x3,y3,z3),···}}

[0025] Where Service refers to the edge server device, and P id Represents the edge server device identifier, α i This represents the parameter information of the i-th camera device, (x i ,y i ) represents the latitude and longitude coordinates of the i-th camera device; z i This represents the data collected by the i-th camera device;

[0026] The cloud server stores an edge server device table W, which records the areas where people have visited. The storage format of the edge server device table W is as follows:

[0027] W = {P id :{person id :{time1,time2,…}},…}

[0028] Among them, P id The person represents the edge server device identifier. id Indicates a person identifier, time k Indicates time information.

[0029] Optionally, confirming that someone is the same person based on the feature matrix and a pre-established database of personal feature information includes:

[0030] The characteristic matrix Q * Match the data with the character feature database to find the match with the feature matrix Q. * The feature matrix of the nearest distance is used to determine whether the person appearing in different camera devices is the same person.

[0031] Optionally, the method for drawing the pedestrian trajectory includes:

[0032] Query the edge server device table W, and find the corresponding edge server device identifier P based on the person identifier. id And time information;

[0033] Based on the edge server device identifier P id The system calls the camera device table E stored in the edge server, compares it with the person's identifier, and determines the camera's location.

[0034] Based on the time information and camera location, the map API is called to draw the person's trajectory.

[0035] Secondly, this invention provides a method for characterizing pedestrian trajectories that supports cross-domain recognition in an edge intelligence environment, applied to a cloud server, comprising:

[0036] Receive the feature matrix and edge server device identifier sent from the edge device server;

[0037] Based on a database of personal information that matches the personal feature information database, the specific information of the person is determined, and the person's trajectory is drawn based on the specific information of the person and the identifiers of all edge server devices the person has passed through.

[0038] The feature matrix and edge server device identifier are obtained by the edge device server through the following method:

[0039] For the acquired images containing people, the features of the people are extracted using an object detection model;

[0040] The character features are clustered according to a preset feature clustering formula to generate a corresponding feature matrix;

[0041] If the feature matrix and the pre-established database of personal feature information confirm that the person is the same person, then the feature matrix and the edge server device identifier are uploaded to the cloud server together.

[0042] Optionally, the method for constructing the personal feature information database includes:

[0043] For different individuals, pre-defined feature clustering and fusion steps are performed to obtain the corresponding clustering and fusion feature matrix;

[0044] Based on the clustering and fusion feature matrices corresponding to different individuals, a database of individual feature information is constructed.

[0045] The preset feature clustering fusion step includes:

[0046] For images of the same person in different dimensions, the backbone feature extraction network of the object detection model is used to extract the corresponding feature maps, forming Q... all Q all ={Q1,Q2,Q3,Q4,...}, where Q1,Q2,Q3,Q4,... represent feature maps in different dimensions;

[0047] The feature maps under the aforementioned multi-dimensional structure are clustered according to a preset feature clustering formula to obtain the corresponding initial clustering fusion feature matrix U; the mathematical expression of the initial clustering feature matrix U is:

[0048]

[0049] in, The characteristic matrix is ​​obtained by analyzing the characteristic map Q. d Feature compression is performed, followed by local feature aggregation using adaptive average feature pooling; (a, b) is the feature matrix. The eigenvalues ​​in f agg For the adaptive average feature pooling function, T all This represents the feature matrix of all person information processed by the pedestrian re-identification model T, with values ​​ranging from 1 to n. i,j Let represent the feature matrix of the i-th layer and j-th dimension after passing through the pedestrian re-identification model T;

[0050] The initial clustering fusion feature matrix U is compressed using a random forest greedy algorithm to obtain the clustering fusion feature matrix. The expression is:

[0051]

[0052] Where m and n are any two eigenvalues ​​in U.

[0053] Optionally, each edge server device stores a camera device table E to store camera device information. The storage format of the camera device table E is as follows:

[0054] E = {Service:P id ,Camera:{α1(x1,y1,z1),α2(x2,y2,z2),α3(x3,y3,z3),···}}

[0055] Where Service refers to the edge server device, and P id Represents the edge server device identifier, α i This represents the parameter information of the i-th camera device, (x i ,y i ) represents the latitude and longitude coordinates of the i-th camera device; z i This represents the data collected by the i-th camera device;

[0056] The cloud server stores an edge server device table W, which records the areas where people have visited. The storage format of the edge server device table W is as follows:

[0057] W = {P id:{person id :{time1,time2,···}},···}

[0058] Among them, P id The person represents the edge server device identifier. id Indicates a person identifier, time k Indicates time information;

[0059] The method for drawing pedestrian trajectories includes:

[0060] Query the edge server device table W, and find the corresponding edge server device identifier P based on the person identifier. id And time information;

[0061] Based on the edge server device identifier P id The system calls the camera device table E stored in the edge server, compares it with the person's identifier, and determines the camera's location.

[0062] Based on the time information and camera location, the map API is called to draw the person's trajectory.

[0063] Thirdly, the present invention provides a pedestrian trajectory characterization system that supports cross-domain recognition in an edge intelligence environment, including a storage medium and a processor;

[0064] The storage medium is used to store instructions;

[0065] The processor is configured to operate according to the instructions to perform the method according to any one of the first aspects.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0067] This invention proposes a method and system for pedestrian trajectory characterization supporting cross-domain recognition in an edge intelligence environment. For images containing people, a target detection model is used to extract human features. These features are then clustered according to a preset feature clustering formula to generate a corresponding feature matrix. When the feature matrix and a pre-established human feature information database confirm the same person, the feature matrix and edge server device identifiers are uploaded to a cloud server. The cloud server then uses a human information database matching the human feature information database to determine the person's specific information. Based on this information and all edge server device identifiers the person has passed through, the cloud server can draw the pedestrian trajectory, capturing the complete behavioral trajectory of the person and characterizing it on a map. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0069] Figure 1 This is a flowchart illustrating a pedestrian trajectory characterization method according to an embodiment of the present invention;

[0070] Figure 2 This is a data processing diagram of a pedestrian trajectory characterization method according to an embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0072] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0073] A memory is an artificial intelligence model based on pattern matching that performs complex cognitive and reasoning tasks by storing and retrieving a large number of patterns. A memory typically consists of two parts: a storage part and a retrieval part. The storage part usually consists of a large-scale vector or matrix, where each row represents a pattern and each column represents a feature or attribute within the pattern. The storage part can be trained using different learning algorithms to automatically discover and extract key features from patterns and encode these features into vector or matrix form. The retrieval part typically includes a query vector or matrix and a similarity calculator. The query vector or matrix represents the pattern to be retrieved, and the similarity calculator is used to calculate the similarity or distance between the query pattern and the stored patterns. Commonly used similarity calculation methods include cosine similarity, Euclidean distance, and Manhattan distance.

[0074] When a new query pattern arrives, the retrieval unit calculates its similarity with all patterns in the storage unit and returns the top few most similar patterns as the retrieval results. These retrieval results can be further used for tasks such as classification, recognition, and reasoning, thereby achieving person re-identification based on pattern matching.

[0075] Example 1

[0076] This invention provides a method for pedestrian trajectory characterization supporting cross-domain recognition in an edge intelligence environment, applied to edge server devices, such as... Figure 1-2 As shown, it includes the following steps:

[0077] (1) For the obtained images containing people, extract the features of the people using the object detection model;

[0078] (2) Cluster the character features according to the preset feature clustering formula to generate the corresponding feature matrix;

[0079] (3) When the feature matrix and the pre-established person feature information database are confirmed to be the same person, the feature matrix and the edge server device identifier are uploaded to the cloud server together, so that the cloud server determines the specific information of the person based on the person information database that matches the person feature information database, and draws the person's trajectory based on the specific information of the person and all the edge server device identifiers that the person has passed through.

[0080] In practical applications, edge server devices are limited by small storage capacity and slow computing speed. Therefore, it is necessary to preprocess the data acquired by the camera device to reduce the image size at the expense of clarity, without affecting the accuracy of feature extraction. Specifically, firstly, the video file acquired by the camera device is extracted at a rate of one frame per second. The image value is denoted as 's'. Then, the image data is cropped and normalized. The processed image is represented as S, and the expression for S is:

[0081]

[0082] Where min(s) represents the minimum value of the image, and max(s) represents the maximum value of the image.

[0083] In one specific embodiment of the present invention, the target detection model is obtained through the following method:

[0084] The backbone feature extraction network in the YOLOv5 object detection framework is set as Wide-ResNet. Compared to the traditional ResNet, Wide-ResNet is more capable of extracting low-level features while also handling high-level features such as structure and contour. In addition to the backbone network, it also includes a neck network and a head network. The neck network adds several layers between the backbone and head networks to collect feature maps from different stages. The head network is mainly used to predict the type and location of the target.

[0085] When using object detection models to extract features from people, in order to retain enough detailed information without making the extracted abstract information too biased towards the pre-trained data, wide-resNet extracts three layers of feature maps, selecting the second and third layers as the features of the people.

[0086] In one specific embodiment of the present invention, the method for constructing the personal feature information database includes:

[0087] For different individuals, pre-defined feature clustering and fusion steps are performed to obtain the corresponding clustering and fusion feature matrix;

[0088] Based on the clustering and fusion feature matrices corresponding to different individuals, the individual feature information database is constructed.

[0089] The preset feature clustering fusion step includes:

[0090] For different individuals, pre-defined feature clustering and fusion steps are performed to obtain the corresponding clustering and fusion feature matrix;

[0091] A database of personal feature information is constructed based on clustering and fusion feature matrices corresponding to different individuals.

[0092] The preset feature clustering fusion step includes:

[0093] For images of the same person in different dimensions, the backbone feature extraction network of the object detection model is used to extract the corresponding feature maps, forming Q... all Q all ={Q1,Q2,Q3,Q4,...}, where Q1,Q2,Q3,Q4,... represent feature maps in different dimensions;

[0094] The feature maps under the aforementioned multi-dimensional structure are clustered according to a preset feature clustering formula to obtain the corresponding initial clustering fusion feature matrix U; the mathematical expression of the initial clustering feature matrix U is:

[0095]

[0096] in, The characteristic matrix is ​​obtained by analyzing the characteristic map Q. d Feature compression is performed, followed by local feature aggregation using adaptive average feature pooling; (a, b) is the feature matrix. The eigenvalues ​​in f agg For the adaptive average feature pooling function, T all This represents the feature matrix of all person information processed by the pedestrian re-identification model T, with values ​​ranging from 1 to n. i,j This represents the feature matrix of the i-th layer and j-th dimension after passing through the pedestrian re-identification model T; the pedestrian re-identification model T is existing technology and will not be described in detail in this application.

[0097] The initial clustering fusion feature matrix U is compressed using a random forest greedy algorithm to obtain the clustering fusion feature matrix. The expression is:

[0098]

[0099] Where m and n are any two eigenvalues ​​in U.

[0100] In practical applications, the personal information database stores citizens' information such as name, age, and ID number. Because the personal information database involves privacy and personal information security, it is necessary to match the individual's characteristics with the information in the personal information database on the cloud server to complete the matching and recording. During the construction process, the matching between the personal characteristic database and the personal information database is completed on the cloud service. The personal information database is located on the cloud server, while the personal characteristic database is located on an edge server.

[0101] The process of constructing the database of personal characteristics is as follows:

[0102] First, the wide-resNet backbone feature extraction network of the object detection model pre-extracts multi-dimensional human features and forms an information set Q. all ,in:

[0103] Q all ={Q1,Q2,Q3,Q4,······}

[0104] The extracted human features are subjected to multiple feature clusterings of different dimensions. The aim is to reduce the amount of feature information without reducing the accuracy of feature classification, thereby constructing a human feature information database to determine whether the human in multiple videos is the same person, thus completing the task of pedestrian re-identification.

[0105] Then, the character feature information database and the character information database are matched one-to-one to determine the specific information of the character.

[0106] The process of performing multiple feature clustering operations on the extracted human features across different dimensions is as follows:

[0107] The cloud server pre-collects human features, and obtains a feature information set Q by collecting multi-dimensional information about the human. all For each feature map Q, feature compression is performed according to the set parameter patchsize (i.e., the size of the image patch, which defaults to 3). Then, local feature aggregation is performed through adaptive average feature pooling to obtain the final feature map. Let f be a feature matrix with height h, width w, and patch size p in the feature dimension, where (a, b) are the eigenvalues ​​in the feature matrix. agg For the adaptive average feature pooling function, T all This represents the feature matrix of all person information processed by the pedestrian re-identification model T, with values ​​ranging from 1 to n. i,j Let represent the feature matrix of the i-th layer and j-th dimension after passing through the pedestrian re-identification model T. The feature set U after feature aggregation is represented as:

[0108]

[0109] Because the features of each character are analyzed across multiple dimensions, storing each character's features would be extremely expensive. Therefore, further compression of the feature information for each character is achieved using a random forest greedy algorithm. The goal is to find a subset... Make the features on the test set and The feature distance relationship and the feature distance relationship on the test set are consistent with the feature distance relationship of U, thereby reducing the total amount of features that need to be stored. The expression for the final extracted subset of U is:

[0110]

[0111] Where m and n are any two eigenvalues ​​in U, the above formula means to calculate the distance between two eigenvalues, with the aim of finding a point whose distance is approximately equal to that of its surrounding points, so that this one value can replace all the surrounding values.

[0112] The above describes the process of collecting pedestrian information and building a feature database on the cloud server side. The extracted features are stored in the database and indexed to prepare for subsequent pedestrian re-identification. During the monitoring process, the edge server waits for pedestrian features Q collected by the edge cameras, and extracts features according to the aforementioned feature aggregation formula to obtain the feature matrix Q. * Q * Unlike U, Q *The feature matrix Q is obtained by feature aggregation from a single image, while the feature set U is obtained by feature aggregation from multiple images. This makes the feature matrix Q... * Match the data with the character feature database to find the match with the feature matrix Q. * The system identifies a person by identifying the closest feature and records relevant parameters such as camera ID, time, and person ID locally. Then, it transmits the index value to the cloud server to determine the person's detailed information and record the relevant parameters.

[0113] When performing feature matching, the KNN formula is used, and its expression is as follows:

[0114]

[0115] Among them, L p x represents the distance between two points. i For the test value, x j Let p = 2 be the true value. The formula above is used to find the true value closest to the test value, and then the true value is used to confirm the person's identity. Finally, the YOLOv5 detection framework is used to annotate the person's information back into the video and save it.

[0116] In one specific embodiment of the present invention, due to the large number of edge server devices, iterating through all edge server devices sequentially each time a pedestrian trajectory is queried would be time-consuming and incur high maintenance costs. To address these issues, each edge server device stores a camera device table E to store camera device information. The storage format of the camera device table E is as follows:

[0117] E = {Service:P id ,Camera:{α1(x1,y1,z1),α2(x2,y2,z2),α3(x3,y3,z3),···}}

[0118] Where Service refers to the edge server device, and P id Represents the edge server device identifier, α i This represents the parameter information of the i-th camera device, (x i ,y i ) represents the latitude and longitude coordinates of the i-th camera device; z i This represents the data collected by the i-th camera device;

[0119] In one specific embodiment of the present invention, confirming that the person is the same based on the feature matrix and a pre-established database of personal feature information includes:

[0120] The characteristic matrix Q *Match the data with the character feature database to find the match with the feature matrix Q. * The feature matrix of the nearest distance is used to determine whether the person appearing in different camera devices is the same person.

[0121] An edge server device table W is stored in the cloud server to record the areas where people have appeared. The storage format of the edge server device table W is as follows:

[0122] W = {P id :{person id :{time1,time2,···}},···}

[0123] Among them, P id The person represents the edge server device identifier. id Indicates a person identifier, time k Indicates time information.

[0124] Based on this, it is possible to determine the approximate area and time of a person's appearance by accessing the edge server device table W during the query process, and then using the edge server device identifier (P) id Person identifier id By using data from the time and camera device table E, the complete route information of a person's appearance can be determined, greatly reducing the bandwidth consumption caused by large amounts of data transmission and reducing a large amount of redundant storage space. At the same time, it can determine the location information of a person's appearance more quickly.

[0125] In one specific embodiment of the present invention, the method for drawing pedestrian trajectories includes:

[0126] Query the edge server device table W, and find the corresponding edge server device identifier P based on the person identifier. id And time information;

[0127] Based on the edge server device identifier P id The system calls the camera device table E stored in the edge server, compares it with the person's identifier, and determines the camera's location.

[0128] Based on the time information and camera location, the map API is called to draw the person's trajectory.

[0129] The method in the embodiments of the present invention will be described in detail below with reference to a specific implementation method.

[0130] This embodiment will demonstrate the process of pedestrian re-identification and trajectory characterization, as well as the relevant parameter settings.

[0131] The parameters involved on the edge server are shown in Table 1.

[0132] Table 1 shows the parameter settings involved in the method execution process.

[0133]

[0134] In the simulation experiment, it is assumed that there are 9 edge devices. Each device maintains a device dictionary to store information about its nearest neighbors. For example, in edge device A, its ID is 0001, and its nearest neighbors are B and C, with IDs of 0002 and 0003 respectively. An example of information stored in A's dictionary is as follows: {0001:(24°32'00.00"N, 117°22'00.00"E), 0002:(24°32'00.09"N, 117°22'00.14"E), 0002:(24°32'00.14"N, 117°22'00.06"E)}. In the simulation experiment, the original data is segmented into frames per second, then proportionally cropped to (3, 224, 224) for feature extraction, uploaded to a cloud server for classification, and the final result is obtained. The person's data is encrypted to return a token string used as a person's identifier, and the person's related information is stored in the edge server. The stored data is as follows: {'908978677SDAC2131F':[(16, 24, 7, 3), '20230731112036'], '43445234CAR64E':

[0135] [(174, 99, 6, 2), '20230731112036']}. For subsequent data processing, if the same pedestrian is encountered, their coordinates and time information can be added using an identifier. By calculating the offset between the pedestrian's location and the camera's location over different time periods, the pedestrian's direction of travel can be determined. This data can be stored in an array and sequentially recorded as [A, C, E], indicating that the pedestrian traveled from camera A to camera C and was ultimately not detected in the vicinity of camera E. This indicates that the pedestrian's destination was E. Using this data, the pedestrian's trajectory can be plotted on a map.

[0136] Example 2

[0137] This invention provides a method for pedestrian trajectory characterization supporting cross-domain recognition in an edge intelligence environment, applied to a cloud server, such as... Figure 1-2 As shown, it includes the following steps:

[0138] Receive the feature matrix and edge server device identifier sent from the edge device server;

[0139] Based on a database of personal information that matches the personal feature information database, the specific information of the person is determined, and the person's trajectory is drawn based on the specific information of the person and the identifiers of all edge server devices the person has passed through.

[0140] The feature matrix and edge server device identifier are obtained by the edge device server through the following method:

[0141] For the acquired images containing people, the features of the people are extracted using an object detection model;

[0142] The character features are clustered according to a preset feature clustering formula to generate a corresponding feature matrix;

[0143] If the feature matrix and the pre-established database of personal feature information confirm that the person is the same, then the feature matrix and the edge server device identifier are uploaded to the cloud server together.

[0144] In one specific embodiment of the present invention, the method for constructing the personal feature information database includes:

[0145] For different individuals, pre-defined feature clustering and fusion steps are performed to obtain the corresponding clustering and fusion feature matrix;

[0146] Based on the clustering and fusion feature matrices corresponding to different individuals, the individual feature information database is constructed.

[0147] The preset feature clustering fusion step includes:

[0148] For images of the same person in different dimensions, the backbone feature extraction network of the object detection model is used to extract the corresponding feature maps, forming Q... all Q all ={Q1,Q2,Q3,Q4,...}, where Q1,Q2,Q3,Q4,... represent feature maps in different dimensions;

[0149] The feature maps under the aforementioned multi-dimensional structure are clustered according to a preset feature clustering formula to obtain the corresponding initial clustering fusion feature matrix U; the mathematical expression of the initial clustering feature matrix U is:

[0150]

[0151] in, The characteristic matrix is ​​obtained by analyzing the characteristic map Q. d Feature compression is performed, followed by local feature aggregation using adaptive average feature pooling; (a, b) is the feature matrix. The eigenvalues ​​in f aggFor the adaptive average feature pooling function, T all This represents the feature matrix of all person information processed by the pedestrian re-identification model T, with values ​​ranging from 1 to n. i,j Let represent the feature matrix of the i-th layer and j-th dimension after passing through the pedestrian re-identification model T;

[0152] The initial clustering fusion feature matrix U is compressed using a random forest greedy algorithm to obtain the clustering fusion feature matrix. The expression is:

[0153]

[0154] Where m and n are any two eigenvalues ​​in U.

[0155] In one specific embodiment of the present invention, each edge server device stores a camera device table E to store camera device information. The storage format of the camera device table E is as follows:

[0156] E = {Service:P id ,Camera:{α1(x1,y1,z1),α2(x2,y2,z2),α3(x3,y3,z3),···}}

[0157] Where Service refers to the edge server device, and P id Represents the edge server device identifier, α i This represents the parameter information of the i-th camera device, (x i ,y i ) represents the latitude and longitude coordinates of the i-th camera device; z i This represents the data collected by the i-th camera device;

[0158] In one specific embodiment of the present invention, confirming that the person is the same based on the feature matrix and a pre-established database of personal feature information includes:

[0159] The characteristic matrix Q * Match the data with the character feature database to find the match with the feature matrix Q. * The feature matrix of the nearest distance is used to determine whether the person appearing in different camera devices is the same person.

[0160] The cloud server stores an edge server device table W, which records the areas where people have visited. The storage format of the edge server device table W is as follows:

[0161] W = {P id :{person id :{time1,time2,…}},…}

[0162] Among them, P id The person represents the edge server device identifier. id Indicates a person identifier, time k Indicates time information;

[0163] The method for drawing pedestrian trajectories includes:

[0164] Query the edge server device table W, and find the corresponding edge server device identifier P based on the person identifier. id And time information;

[0165] Based on the edge server device identifier P id The system calls the camera device table E stored in the edge server, compares it with the person's identifier, and determines the camera's location.

[0166] Based on the time information and camera location, the map API is called to draw the person's trajectory.

[0167] Based on this, it is possible to determine the approximate area and time of a person's appearance by accessing the edge server device table W during the query process, and then using the edge server device identifier (P) id Person identifier id By using data from the time and camera device table E, the complete route information of a person's appearance can be determined, greatly reducing the bandwidth consumption caused by large amounts of data transmission and reducing a large amount of redundant storage space. At the same time, it can determine the location information of a person's appearance more quickly.

[0168] The method in the embodiments of the present invention will be described in detail below with reference to a specific implementation method.

[0169] This embodiment will demonstrate the process of pedestrian re-identification and trajectory characterization, as well as the relevant parameter settings.

[0170] The parameters involved on the edge server are shown in Table 1.

[0171] Table 1 shows the parameter settings involved in the method execution process.

[0172]

[0173]

[0174] In the simulation experiment, it is assumed that there are 9 edge devices. Each device maintains a device dictionary to store information about its nearest neighbors. For example, in edge device A, its ID is 0001, and its nearest neighbors are B and C, with IDs of 0002 and 0003 respectively. An example of information stored in A's dictionary is as follows: {0001:(24°32'00.00"N, 117°22'00.00"E), 0002:(24°32'00.09"N, 117°22'00.14"E), 0002:(24°32'00.14"N, 117°22'00.06"E)}. In the simulation experiment, the original data is segmented into frames per second, then proportionally cropped to (3, 224, 224) for feature extraction, uploaded to a cloud server for classification, and the final result is obtained. The person's data is encrypted to return a token string used as a person's identifier, and the person's related information is stored in the edge server. The stored data is as follows: {'908978677SDAC2131F':[(16, 24, 7, 3), '20230731112036'], '43445234CAR64E':

[0175] [(174, 99, 6, 2), '20230731112036']}. For subsequent data processing, if the same pedestrian is encountered, their coordinates and time information can be added using an identifier. By calculating the offset between the pedestrian's location and the camera's location over different time periods, the pedestrian's direction of travel can be determined. This data can be stored in an array and sequentially recorded as [A, C, E], indicating that the pedestrian traveled from camera A to camera C and was ultimately not detected in the vicinity of camera E. This indicates that the pedestrian's destination was E. Using this data, the pedestrian's trajectory can be plotted on a map.

[0176] Example 3

[0177] This invention provides a pedestrian trajectory characterization system supporting cross-domain recognition in an edge intelligence environment, including a storage medium and a processor;

[0178] The storage medium is used to store instructions;

[0179] The processor is configured to operate according to the instructions to execute the method according to any one of Embodiment 1 or Embodiment 2.

[0180] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0181] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0182] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0183] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0184] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

[0185] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for characterizing pedestrian trajectories supporting cross-domain recognition in an edge intelligence environment, characterized in that, Applications in edge server devices, including: For the acquired images containing people, the features of the people are extracted using an object detection model; The character features are clustered according to a preset feature clustering formula to generate a corresponding feature matrix; If the same person is identified based on the feature matrix and the pre-established person feature information database, the feature matrix and the edge server device identifier are uploaded to the cloud server together. The cloud server then determines the person's specific information based on the person information database that matches the person feature information database, and draws the person's trajectory based on the person's specific information and all the edge server device identifiers that the person has passed through. The method for constructing the personal characteristic information database includes: For different individuals, pre-defined feature clustering and fusion steps are performed to obtain the corresponding clustering and fusion feature matrix; A database of personal feature information is constructed based on clustering and fusion feature matrices corresponding to different individuals. The preset feature clustering fusion step includes: For images of the same person in different dimensions, the backbone feature extraction network of the object detection model is used to extract the corresponding feature maps, forming... , ,in, These represent feature maps in different dimensions; For feature maps in different dimensions, clustering is performed according to a preset feature clustering formula to obtain the corresponding initial clustering fusion feature matrix. The initial clustering fusion feature matrix The mathematical expression is: in, The feature matrix is ​​obtained by analyzing the feature map. Feature compression is performed, followed by local feature aggregation using adaptive average feature pooling; (a, b) is the feature matrix. eigenvalues ​​in It is an adaptive average feature pooling function. This indicates all pedestrians who have passed through the pedestrian re-identification model. The feature matrix of the person's information, with values ​​ranging from... , This indicates that the pedestrian re-identification model has been used. The Layer, First A feature matrix with multiple dimensions; Using the random forest greedy algorithm, the initial clustering fusion feature matrix is ​​processed. Compression is performed to obtain the clustering fusion feature matrix. , The expression is: ； in, , yes Any two eigenvalues ​​in; Each edge server device stores a camera device table E, which is used to store camera device information. The storage format of the camera device table E is as follows: in, For edge server devices, This represents the edge server device identifier. Indicates the first Parameter information of each camera device Indicates the first Latitude and longitude coordinates of each camera device; Indicates the first Data collected by a single camera device; The cloud server stores a table of edge server devices. This is used to record the areas where people have been; it's a table of edge server devices. The storage format is as follows: in, This represents the edge server device identifier. Represents a person's identifier. Indicates time information; The method for drawing pedestrian trajectories includes: Query the edge server device table Find the corresponding edge server device identifier based on the person identifier. And time information; Based on edge server device identifier The system calls the camera device table E stored in the edge server, compares it with the person's identifier, and determines the camera's location. Based on the time information and camera location, the map API is called to draw the person's trajectory.

2. The method for pedestrian trajectory characterization supporting cross-domain recognition in an edge intelligent environment according to claim 1, characterized in that: The target detection model was obtained through the following method: Set the backbone feature extraction network in the YOLOv5 object detection framework to wide-resNet; When using object detection models to extract features from people, wide-resNet extracts three layers of feature maps, selecting the second and third layers as the features of the person.

3. The method for pedestrian trajectory characterization supporting cross-domain recognition in an edge intelligence environment according to claim 1, characterized in that: Based on the feature matrix and a pre-established database of personal feature information, it was confirmed that they are the same person, including: feature matrix Match the data with the character feature database to find the match with the feature matrix Q. * The feature matrix of the nearest distance is used to determine whether the person appearing in different camera devices is the same person.

4. A method for characterizing pedestrian trajectories that supports cross-domain recognition in an edge intelligence environment, characterized in that, Applied to cloud servers, including: Receive the feature matrix and edge server device identifier sent from the edge device server; Based on a database of personal information that matches the personal feature information database, the specific information of the person is determined, and the person's trajectory is drawn based on the specific information of the person and the identifiers of all edge server devices the person has passed through. The feature matrix and edge server device identifier are obtained by the edge device server through the following method: For the acquired images containing people, the features of the people are extracted using an object detection model; The character features are clustered according to a preset feature clustering formula to generate a corresponding feature matrix; If the feature matrix and the pre-established database of personal feature information confirm that the person is the same person, then the feature matrix and the edge server device identifier are uploaded to the cloud server together. The method for constructing the personal characteristic information database includes: For different individuals, pre-defined feature clustering and fusion steps are performed to obtain the corresponding clustering and fusion feature matrix; A database of personal feature information is constructed based on clustering and fusion feature matrices corresponding to different individuals. The preset feature clustering fusion step includes: For images of the same person in different dimensions, the backbone feature extraction network of the object detection model is used to extract the corresponding feature maps, forming... , ,in, These represent feature maps in different dimensions; For feature maps in different dimensions, clustering is performed according to a preset feature clustering formula to obtain the corresponding initial clustering fusion feature matrix. The initial clustering fusion feature matrix The mathematical expression is: in, The feature matrix is ​​obtained by analyzing the feature map. Feature compression is performed, followed by local feature aggregation using adaptive average feature pooling; (a, b) is the feature matrix. eigenvalues ​​in It is an adaptive average feature pooling function. This indicates all pedestrians who have passed through the pedestrian re-identification model. The feature matrix of the person's information, with values ​​ranging from... , This indicates that the pedestrian re-identification model has been used. The Layer, First A feature matrix with multiple dimensions; Using the random forest greedy algorithm, the initial clustering fusion feature matrix is ​​processed. Compression is performed to obtain the clustering fusion feature matrix. , The expression is: ； in, , yes Any two eigenvalues ​​in; The feature is that each edge server device stores a camera device table E, which is used to store camera device information. The storage format of the camera device table E is as follows: in, For edge server devices, This represents the edge server device identifier. Indicates the first Parameter information of each camera device Indicates the first Latitude and longitude coordinates of each camera device; Indicates the first Data collected by a single camera device; The cloud server stores a table of edge server devices. This is used to record the areas where people have been; it's a table of edge server devices. The storage format is as follows: in, This represents the edge server device identifier. Represents a person's identifier. Indicates time information; The method for drawing pedestrian trajectories includes: Query the edge server device table Find the corresponding edge server device identifier based on the person identifier. And time information; Based on edge server device identifier The system calls the camera device table E stored in the edge server, compares it with the person's identifier, and determines the camera's location. Based on the time information and camera location, the map API is called to draw the person's trajectory.

5. A pedestrian trajectory characterization system supporting cross-domain recognition in an edge intelligence environment, characterized in that, Including storage media and processor; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the method according to any one of claims 1-3.

Images