Risk personnel feature analysis method and storage medium

Abstract

The invention discloses a feature analysis method for risk personnel and a storage medium. The method comprises the following steps: step 1, storing personnel basic information and corresponding trajectory data into a data platform; 2, cleaning and classifying the data, and classifying and storing the data according to static and dynamic data; 3, extracting data features, and analyzing the data features by using a feature analysis algorithm to obtain a weight value and a weight sum of each feature of the personnel; 4, sorting is carried out based on the weight sum of the personnel, feature data of the corresponding personnel is extracted from the personnel of which the weight sum ranks top x% and marked as positive samples, and corresponding feature data is extracted from other personnel and marked as negative samples; 5, using the extracted feature data as input features, and inputting the input features into a machine learning model for training to obtain a trained machine learning model; and step 6, analyzing the feature data of the new input personnel through the trained model, and outputting the probability that the feature data of the new input personnel are positive samples.

Description

Technical Field

[0001] This invention relates to the field of big data analytics, specifically to a method and storage medium for analyzing the characteristics of at-risk individuals. Background Technology

[0002] With the rapid development of modern society, economy, and technology, urban sensing equipment is constantly improving, and massive amounts of multi-dimensional personnel and trajectory data are converging into information management systems. This data possesses enormous potential for data mining. Traditional data analysis methods rely on the work experience of relevant personnel to judge the possible trajectory tendencies of those under observation. This method has the following drawbacks:

[0003] 1. Relying on personal experience to make judgments and conduct investigations is time-consuming and labor-intensive, and not widely applicable;

[0004] 2. Manual investigation is required to track the movements of people and vehicles of individuals under surveillance and their related persons, which may result in human error.

[0005] Therefore, it is necessary to develop a characteristic analysis method for risk personnel based on existing data mining. Summary of the Invention

[0006] In view of the technical defects and drawbacks existing in the prior art, embodiments of the present invention provide a method and storage medium for characterizing risk personnel to overcome or at least partially solve the above problems. The specific solution is as follows:

[0007] As a first aspect of the present invention, a method for analyzing the characteristics of risk personnel is provided, the method comprising:

[0008] Step 1: Store basic personnel information and corresponding trajectory data into the data platform;

[0009] Step 2: Clean and classify the data stored in the data platform, and store the data according to static and dynamic data categories;

[0010] Step 3: Extract static and dynamic data features from static and dynamic data, use feature analysis algorithms to analyze the static and dynamic data features, obtain the weight value of each feature of the personnel, and obtain the weight sum of individual personnel;

[0011] Step 4: Sort the personnel based on their weights and rankings. For the top x% of personnel by weight and ranking, extract their feature data and label them as positive samples. For the other personnel, extract their feature data and label them as negative samples.

[0012] Step 5: Use the extracted feature data as input features to train the machine learning model and obtain the trained machine learning model;

[0013] Step 6: Analyze the feature data of the new input personnel using the trained model, and output the probability that the feature data of the new input personnel is a positive sample.

[0014] Furthermore, in step 1, the basic information of the personnel includes name, ID number, contact information, household registration information, information of temporary residents, information of people entering the facility, information of the floating population, social security payment information and employment information, and the trajectory data includes facial capture trajectory, vehicle capture trajectory (including facial recognition information of front-seat passengers) and hotel check-in information.

[0015] Furthermore, in step 2, data cleaning includes data deduplication, removal of null values, and cleaning of dirty data.

[0016] Furthermore, in step 2, static data is person-based tag data, including seven tags: local household registration, temporary residents, inmates, migrant population, first-time visitors, those without local social security, and special industry practitioners; dynamic data is person-behavior-based tag data, including eight tags: apartment snapshots, multiple unregistered stays, multiple stays, multiple consecutive stays, multiple trips in different vehicles, snapshots of the same hotel as inmates, number of times traveling with inmates, and number of times traveling with different members of the opposite sex.

[0017] Furthermore, in step 3, the feature analysis algorithm used is the TF-IDF algorithm.

[0018] Furthermore, feature analysis algorithms are used to analyze the static and dynamic data features to obtain the weight value of each feature of the personnel, and the weight of each individual personnel is obtained, specifically including:

[0019] After calculating using TF, the representativeness of the feature is obtained, expressed by the following formula:

[0020]

[0021] in , This represents the total number of times a certain characteristic of person p appears. This represents the total number of times all characteristics of person p appear.

[0022] The discriminant strength of the features is obtained after IDF calculation;

[0023]

[0024] in This represents the total number of occurrences of all target personnel characteristics;

[0025] Based on TF and IDF calculations, the steps for calculating the weight values ​​of individual features of the final target personnel are as follows:

[0026]

[0027] in, It represents the total number of times a certain feature t appears in all people (sample set), that is, how many people have feature t;

[0028] Similarly, based on the weight value of each feature of the target personnel, the weight value of each feature of each target personnel is calculated. Based on the weight values ​​of each feature of the target personnel, the weight sum of each target personnel is obtained. The calculation formula is as follows:

[0029]

[0030] a and b are the slope coefficient and intercept coefficient of the linear correlation.

[0031] Furthermore, in step 4, the feature data consists of a two-dimensional array, where each row represents a sample, each column represents a feature, and the values ​​in the array are the weight values ​​of the features corresponding to the sample.

[0032] Furthermore, in step 4, the number of positive samples is 1, and the number of negative samples is 0.

[0033] Furthermore, in step 5, the machine learning model is the XGBoost model, which is divided into two processes: training and prediction. The training process includes initializing model parameters, building an initial model, and iterative optimization; the prediction process includes input data, model prediction, and generating prediction results; the initial model parameters are a learning rate of 5%, a maximum depth of 3, 20 training rounds, and a positive to negative sample ratio of 1:10 in the training set.

[0034] As a second aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a computer, the computer program causes the computer to perform the risk person characterization method as described above.

[0035] The present invention has the following beneficial effects:

[0036] This invention provides a feature analysis method for risky individuals. By analyzing basic information and corresponding trajectory data, it obtains the static and dynamic data features of the corresponding individuals. A feature analysis algorithm is then used to analyze these static and dynamic features, obtaining the weight value and weight sum of each feature. The weight sum is used to confirm the sample type of the feature data for each individual. Finally, a trained machine learning model is used to analyze the feature data of newly input individuals, outputting the probability that the feature data of the newly input individual is a positive sample. This invention fully utilizes existing multidimensional data. By analyzing multidimensional data and combining it with learning model training, it provides reasonable predictions of target events for risky individuals. This analytical method can further reference the recent movements of risky individuals, providing early warnings and recommendations for investigation scope, offering valuable reference for practitioners in related industries to innovate analytical perspectives. Attached Figure Description

[0037] Figure 1 A flowchart illustrating the risk personnel characteristic analysis method provided in this embodiment of the invention;

[0038] Figure 2 This is a schematic diagram illustrating the training results of the model provided in an embodiment of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] like Figure 1 As shown, this invention provides a method for analyzing the characteristics of risk personnel, the method comprising:

[0041] Step 1: Store basic personnel information and corresponding trajectory data into the data platform;

[0042] Step 2: Clean and classify the data stored in the data platform, and store the data according to static and dynamic data categories;

[0043] Step 3: Extract static and dynamic data features from static and dynamic data, use feature analysis algorithms to analyze the static and dynamic data features, obtain the weight value of each feature of the personnel, and obtain the weight sum of individual personnel;

[0044] Step 4: Sort the personnel based on their weights and rankings. For the top x% of personnel by weight and ranking, extract their feature data and label them as positive samples. For the other personnel, extract their feature data and label them as negative samples.

[0045] Step 5: Use the extracted feature data as input features to train the machine learning model and obtain the trained machine learning model;

[0046] Step 6: Analyze the feature data of the new input personnel using the trained model, and output the probability that the feature data of the new input personnel is a positive sample.

[0047] This invention makes full use of existing multidimensional data. By analyzing multidimensional data and training a learning model, it provides reasonable predictions of target events for at-risk individuals. Through this analytical method, it can further consider the recent movements of at-risk individuals, providing early warnings and recommendations for investigation scope, thus offering valuable reference for practitioners in related industries to innovate their analytical perspectives.

[0048] In some embodiments, the basic information of the personnel includes name, ID number, contact information, household registration information, information of temporary residents, information of people entering the facility, information of the floating population, social security payment information and employment information, etc., and the trajectory data includes facial capture trajectory, vehicle capture trajectory (including facial recognition information of front-seat passengers) and hotel check-in information, etc.

[0049] In some embodiments, cleaning the data stored in the data platform includes: performing operations such as deduplication, removing null values, and cleaning dirty data on the basic information of personnel and corresponding trajectory data within the data platform.

[0050] In some embodiments, static data is person-based tag data, including seven types of tag data: local household registration, temporary residents, inmates, migrant population, first-time visitors, no local social security, and special industry practitioners; dynamic data is person-behavior-based tag data, including eight types of tag data: apartment snapshots, multiple unregistered stays, multiple stays, multiple consecutive stays, multiple rides in different vehicles, snapshots of the same hotel as inmates, number of times traveling with inmates, and number of times traveling with different members of the opposite sex.

[0051] It is important to note that all of the above static and dynamic data sources are legal.

[0052] In some embodiments, the eigenvalue analysis algorithm used is the TF-IDF algorithm, wherein the TF-IDF algorithm includes the TF algorithm and the IDF algorithm;

[0053] The TF algorithm represents the frequency of a feature value in the overall feature value system. The higher the frequency, the more representative the feature value is of the overall sample of people of interest, and the higher the weight should be assigned to the feature value. After TF calculation, the representativeness of the feature value is obtained, as shown in the following formula:

[0054]

[0055] , This represents the total number of times a certain characteristic of person p appears. This represents the total number of times all characteristics of person p appear.

[0056] The IDF (Inverse Eigenvalue Frequency) algorithm posits that the lower the frequency of a particular eigenvalue in the total eigenvalues, the stronger its discriminative power and the closer its correlation with the corresponding individuals. Even if a eigenvalue appears with the lowest frequency in the total sample, experience suggests that such individuals often have higher risk factors, indicating that the eigenvalue's discriminative power is greater. Therefore, it's crucial to highlight individual eigenvalues ​​and their weighting. The discriminative power of an eigenvalue is calculated using the IDF algorithm, expressed as follows:

[0057]

[0058] in, This represents the total number of occurrences of all target personnel feature values.

[0059] The TF-IDF algorithm posits that the lower the frequency of a feature value, the greater its ability to distinguish other samples; conversely, the higher the frequency of a feature value in the total sample, the lower its IDF should be. By analyzing a specific high-frequency and low-frequency feature value within the sample and performing a neutralization calculation, a relatively balanced TF-IDF weight value can be obtained.

[0060] The final weight values ​​for individual features are calculated as follows:

[0061]

[0062] in, This represents the total number of times a certain feature t appears among all people;

[0063] After calculating the weight of a single feature value for a person, the weight percentage of each feature value for each target person can be obtained by analogy. This allows us to further calculate the sum of the weights for each target person, as shown in the following formula:

[0064]

[0065] Where a and b are the slope coefficient and intercept coefficient of the linear correlation.

[0066] By following the above method, the weight values ​​of all target personnel samples can be obtained. Based on the weight values, the personnel in the samples can be compared. Samples with higher weights have higher specificity and need to be given special attention. Combined with the local target event occurrence rate, the samples can be divided into positive samples and negative samples.

[0067] For example, if the probability of a local comprehensive target event occurring is x%, then the samples with the highest weight ranking (top x%) can be considered to have a higher probability of occurrence and are marked as positive samples, while those with lower weight ranking are marked as negative samples.

[0068] Using the extracted feature data as input features, a machine learning model is trained to obtain a model that can be used to predict the probability of the occurrence of a target event. The trained model can then analyze new input parameters (feature data of new input personnel) and output the probability that the new sample is a positive sample, i.e., the probability of the target event occurring.

[0069] In some embodiments, the feature data consists of a two-dimensional array, where each row represents a sample (i.e., a person) and each column represents a feature (i.e., a feature of the person). The values ​​in the array are the weight values ​​of the features corresponding to the sample. In step 5, the model fitting target is the sample type, where positive samples are 1 and negative samples are 0.

[0070] In some embodiments, the machine learning model refers to the XGBoost model, which consists of two processes: training and prediction. The training process includes initializing model parameters, building an initial model, and iterative optimization. The prediction process includes input data, model prediction, and generating prediction results. The initial model parameters are a learning rate of 5%, a maximum depth of 3, 20 training epochs, and a positive-to-negative sample ratio of 1:10 in the training set. The training results are as follows. Figure 2 As shown.

[0071] Overfitting can be avoided and prediction accuracy improved by adjusting the parameter values ​​during XGBoost training. The training results of the model are as follows. Figure 2 As shown, after 20 rounds of training, the prediction results have converged and the accuracy is stable and usable.

[0072] This invention also provides a computer-readable storage medium storing a computer program. When executed by a computer, the computer program causes the computer to perform the risk personnel characteristic analysis method as described above. The risk personnel characteristic analysis method has been described above and will not be repeated here.

[0073] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for analyzing the characteristics of risk personnel, characterized in that, The method includes: Step 1: Store basic personnel information and corresponding trajectory data into the data platform; Step 2: Clean and classify the data stored in the data platform, and store the data according to static and dynamic data categories; Step 3: Extract static and dynamic data features from static and dynamic data, use feature analysis algorithms to analyze the static and dynamic data features, obtain the weight value of each feature of the personnel, and obtain the weight sum of individual personnel; Step 4: Sort the personnel based on their weights and rankings. For the top x% of personnel by weight and ranking, extract their feature data and label them as positive samples. For the other personnel, extract their feature data and label them as negative samples. Step 5: Use the extracted feature data as input features to train the machine learning model and obtain the trained machine learning model; Step 6: Analyze the feature data of the new input personnel using the trained model, and output the probability that the feature data of the new input personnel is a positive sample.

2. The risk personnel characteristic analysis method according to claim 1, characterized in that, In step 1, the basic information of the personnel includes name, ID number, contact information, household registration information, information of temporary residents, information of people entering the facility, information of the floating population, social security payment information and employment information, and trajectory data includes facial capture trajectory, vehicle capture trajectory and hotel check-in information.

3. The risk personnel characteristic analysis method according to claim 1, characterized in that, Step 2 involves cleaning the data, including deduplication, removing null values, and cleaning dirty data.

4. The risk personnel characteristic analysis method according to claim 1, characterized in that, In step 2, static data is person-based tag data, including seven types of tag data: local household registration, temporary residents, inmates, migrant population, first-time visitors, those without local social security, and special industry practitioners; dynamic data is person-behavior-based tag data, including eight types of tag data: apartment snapshots, multiple unregistered stays, multiple stays, multiple consecutive stays, multiple rides in different vehicles, snapshots of the same hotel as inmates, number of times traveling with inmates, and number of times traveling with different members of the opposite sex.

5. The risk personnel characteristic analysis method according to claim 1, characterized in that, In step 3, the feature analysis algorithm used is the TF-IDF algorithm.

6. The risk personnel characteristic analysis method according to claim 5, characterized in that, Feature analysis algorithms are used to analyze static and dynamic data features to obtain the weight value of each feature of a person, and to obtain the weight of an individual person, including: After calculating using TF, the representativeness of the feature is obtained, expressed by the following formula: w(p,t)=∑x i Where p∈target personnel, x i Let w(p,t) represent the total number of times a certain feature of person p occurs, where ∑w(p,t) is the i-th feature value. i () represents the total number of times all characteristics of person p appear; The discriminant strength of the features is obtained after IDF calculation; Wherein, ΣΣw(p j ,t i () represents the total number of occurrences of all target personnel characteristics; Based on TF and IDF calculations, the steps for calculating the weight values ​​of individual features of the final target personnel are as follows: Wherein, Σw(p j (t) represents the total number of times a certain feature t appears among all people; Similarly, based on the weight values ​​of individual features of each target person, the weight value of each feature of each target person is calculated. Based on the weight values ​​of each feature of each target person, the weight sum f(k) of each target person is obtained. The calculation formula is as follows: f(k)=a∑k i +b Where a and b are the slope coefficient and intercept coefficient of the linear correlation.

7. The method for analyzing the characteristics of risk personnel according to claim 1, characterized in that, In step 4, the feature data consists of a two-dimensional array, where each row represents a sample, each column represents a feature, and the values ​​in the array are the weight values ​​of the features corresponding to the sample.

8. The method for analyzing the characteristics of risk personnel according to claim 1, characterized in that, In step 4, the number of positive samples is 1, and the number of negative samples is 0.

9. The method for analyzing the characteristics of risk personnel according to claim 1, characterized in that, In step 5, the machine learning model is the XGBoost model, which consists of two processes: training and prediction. The training process includes initializing model parameters, building an initial model, and iterative optimization. The prediction process includes input data, model prediction, and generating prediction results. The initial parameters of the model are a learning rate of 5%, a maximum depth of 3, 20 training epochs, and a positive to negative sample ratio of 1:10 in the training set.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a computer, causes the computer to perform the risk personnel characterization method as described in any one of claims 1 to 9.

Images