Identification model training method and device, work order identification method and device, medium and equipment

Abstract

The embodiment of the invention provides an identification model training method and a work order identification method. The method comprises the following steps: acquiring sample work order data of a sample work order and a preset random forest model; performing data cleaning and feature extraction on the sample work order data to obtain sample work order features after noise influence is removed; dividing the sample work order features into a plurality of sample sets; for each sample set, calculating the importance score of a sample feature cluster to which the sample work order feature belongs, calculating the relevancy between the work order feature and other sample work order features in the corresponding sample feature cluster, and sorting the sample work order features according to the relevancy and the importance score of the sample feature cluster; obtaining a target sample work order feature sequence; and training a preset random forest model according to the target sample work order feature sequence and a sample result marked on the sample work order to obtain an abnormal work order identification model. According to the invention, the identification model can be trained.

Description

Technical Field

[0001] The present invention belongs to the technical field of communication services, and in particular relates to a recognition model training method, a work order recognition method, a device, a computer storage medium and an electronic device. Background Art

[0002] In the industry, installation and maintenance services are the core link to ensure network stability and user satisfaction. Installation and maintenance technicians play a vital role in this process, responsible for equipment installation, technical debugging, troubleshooting and collecting customer feedback. Standardized work orders allow operators to accurately grasp the various equipment network elements, line cutover conditions and fault conditions of various specialties in various cities, so as to ensure network stability and security and improve user perception.

[0003] In related technologies, the number of work orders generated by network management systems of different professions and work order systems of different cities is huge, involving multiple professions. In addition, the personal experience and sense of responsibility of operation and maintenance personnel in cities are different, and the degree of communication with fault handling personnel is different, which usually generates irregular or abnormal work orders. For these abnormal work orders, operation and maintenance personnel usually need to manually determine them. This determination method relies on the subjective judgment of operation and maintenance personnel, has low accuracy and low efficiency. Summary of the invention

[0004] The present invention provides a recognition model training method, a work order recognition method, a device, a computer storage medium and an electronic device to solve the problems of low detection efficiency and low accuracy in related technologies.

[0005] In a first aspect, the present invention provides a recognition model training method, the method comprising:

[0006] Obtaining sample work order data of a sample work order and a preset random forest model, wherein the sample work order data is maintenance data information corresponding to a maintenance task performed in the service;

[0007] Performing data cleaning and feature extraction on the sample work order data to obtain sample work order features after removing noise influence;

[0008] Dividing the sample work order features into multiple sample sets;

[0009] For each of the sample sets, the importance score of the sample feature cluster to which the sample work order feature belongs is calculated, and the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster is calculated, and the sample work order features are sorted according to the correlation and the importance score of the sample feature cluster to obtain a target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the status identification of the work order; the correlation represents the correlation between the sample work order feature in the sample feature cluster and the classification information;

[0010] According to the target sample work order feature sequence and the sample results of the sample work order annotation, the preset random forest model is trained to obtain an abnormal work order recognition model. The abnormal work order recognition model is used to determine the recognition result of the work order corresponding to the work order data based on the input work order data. The recognition result is used to include a normal state and an abnormal state.

[0011] In a second aspect, the present invention provides a work order identification method, the method comprising:

[0012] Obtain target work order data of a target work order and a trained abnormal work order recognition model, wherein the target work order data is maintenance data information corresponding to a maintenance task currently being performed in the service;

[0013] Performing data cleaning and feature extraction on the target work order data to obtain target work order features after removing noise influence;

[0014] The target work order feature is input into the abnormal work order recognition model to obtain a recognition result output by the abnormal work order recognition model, wherein the recognition result includes a normal state and an abnormal state.

[0015] In a third aspect, the present invention provides a recognition model training device, the device comprising:

[0016] A first acquisition module is used to acquire sample work order data of a sample work order and a preset random forest model, wherein the sample work order data is maintenance data information corresponding to a maintenance task performed in the service;

[0017] A first cleaning module is used to perform data cleaning and feature extraction on the sample work order data to obtain sample work order features after noise influence is removed;

[0018] A division module, used for dividing the sample work order features into multiple sample sets;

[0019] A determination module is used to calculate, for each of the sample sets, an importance score of the sample feature cluster to which the sample work order feature belongs, and calculate the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster, and sort the sample work order features according to the correlation and the importance score of the sample feature cluster to obtain a target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the status identification of the work order; the correlation represents the correlation between the sample work order feature in the sample feature cluster and the classification information;

[0020] A training module is used to train a preset random forest model according to the target sample work order feature sequence and the sample results of the sample work order annotation to obtain an abnormal work order recognition model. The abnormal work order recognition model is used to determine the recognition result of the work order corresponding to the work order data based on the input work order data, and the recognition result is used to include a normal state and an abnormal state.

[0021] In a fourth aspect, the present invention provides an identification device, the device comprising:

[0022] A second acquisition module is used to acquire target work order data of a target work order and a trained abnormal work order recognition model, wherein the target work order data is maintenance data information corresponding to a maintenance task currently being executed in the service;

[0023] A second cleaning module is used to perform data cleaning and feature extraction on the target work order data to obtain the target work order features after removing the noise influence;

[0024] The recognition module is used to input the target work order feature into the abnormal work order recognition model to obtain the recognition result output by the abnormal work order recognition model, and the recognition result is used to include a normal state and an abnormal state.

[0025] In a fifth aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the methods described in the first and second aspects when executing the program.

[0026] In a sixth aspect, the present invention provides a readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the methods described in the first and second aspects.

[0027] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0028] By obtaining sample work order data of a sample work order and a preset random forest model, the sample work order data is maintenance data information corresponding to the maintenance task performed in the service; performing data cleaning and feature extraction on the sample work order data to obtain sample work order features after removing the influence of noise; dividing the sample work order features into multiple sample sets; for each of the sample sets, calculating the importance score of the sample feature cluster to which the sample work order features belong, and calculating the correlation between the sample work order features and other sample work order features in the corresponding sample feature cluster, and dividing the sample work order features according to the correlation and the sample set. The importance scores of feature clusters are sorted to obtain a target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the state identification of the work order; the correlation represents the correlation between the sample work order features in the sample feature cluster and the classification information; according to the target sample work order feature sequence and the sample results of the sample work order annotation, the preset random forest model is trained to obtain an abnormal work order recognition model, and the abnormal work order recognition model is used to determine the recognition result of the work order corresponding to the work order data according to the input work order data, and the recognition result is used to include normal state and abnormal state. In this way, by using the random forest algorithm to analyze and learn the sample work order data of a large number of sample work orders, abnormal work orders can be accurately identified, the detection accuracy and detection efficiency can be improved, and the problem of low detection efficiency caused by relying on physical inspection or manual inspection in traditional detection methods can be avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 is a flow chart of a recognition model training method provided in an embodiment of the present application;

[0031] Figure 2 is a flow chart of a work order identification method provided in an embodiment of the present application;

[0032] Figure 3 is a processing flow chart of an identification system provided by an embodiment of the present application;

[0033] Figure 4 It is a flow chart of model optimization provided by an embodiment of the present application;

[0034] Figure 5It is a schematic diagram of the execution steps of building, training and applying a recognition model provided in an embodiment of the present application;

[0035] Figure 6 It is a flow chart of a method for determining a target work order feature sequence provided by an embodiment of the present application;

[0036] Figure 7 It is a flow chart of a method for determining characteristics of a sample work order provided in an embodiment of the present application;

[0037] Figure 8 is a block diagram of a recognition model training device provided in an embodiment of the present application;

[0038] Fig. 9 is a block diagram of an identification device provided in an embodiment of the present application;

[0039] Fig.10 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Before introducing the recognition model training method, work order recognition method, device, computer storage medium and electronic device provided by the present disclosure, the application scenarios involved in each embodiment of the present disclosure are first introduced. The present disclosure can be applied to the scenario of detecting installation and maintenance work orders, and the recognition model training method and work order recognition method provided by the embodiments of the present disclosure can be applied to electronic devices.

[0042] In the industry, installation and maintenance services are the core link to ensure network stability and user satisfaction. Installation and maintenance technicians play a vital role in this process, responsible for equipment installation, technical debugging, troubleshooting and collecting customer feedback. Standardized work orders allow operators to accurately grasp the various equipment network elements, line cutover conditions and fault conditions of various specialties in various cities, so as to ensure network stability and security and improve user perception.

[0043] In related technologies, the number of work orders generated by network management systems of different professions and work order systems of different cities is huge, involving multiple professions. In addition, the personal experience and sense of responsibility of operation and maintenance personnel in cities are different, and the degree of communication with fault handling personnel is different, which usually generates irregular or abnormal work orders. For these abnormal work orders, operation and maintenance personnel usually need to manually determine them. This determination method relies on the subjective judgment of operation and maintenance personnel, has low accuracy and low efficiency.

[0044] In order to solve the above problems, the present disclosure provides a recognition model training method, a work order recognition method, a device, a computer storage medium and an electronic device, which can analyze and learn sample work order data of a large number of sample work orders by adopting a random forest algorithm, accurately identify abnormal work orders, improve the detection accuracy and detection efficiency, and avoid the problem of low detection efficiency caused by traditional detection methods relying on physical inspections or manual inspections.

[0045] The following is a detailed description of the recognition model training method provided in the embodiment of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0046] Figure 1 is a flow chart of a recognition model training method provided in an embodiment of the present application, such as Figure 1 As shown, the method may include the following steps.

[0047] In step S101, sample work order data of a sample work order and a preset random forest model are obtained.

[0048] Among them, the sample work order data is the maintenance data information corresponding to the maintenance tasks performed in the service. The sample work order may refer to a document used in the industry to record, track and manage service requests such as equipment maintenance, troubleshooting, and product replacement. It can be used to centrally manage customer requests, automatically dispatch work orders, track processing status, and provide data analysis and reporting functions, which can help companies improve the efficiency and quality of after-sales service.

[0049] In this step, maintenance data information corresponding to historical maintenance tasks can be comprehensively collected from channels such as the online business hall, backend database, integrated dispatching system, customer dispatching and customer information system, etc. The maintenance data information may include login logs, operation records, order details, IP address (Internet Protocol Address), equipment information, installation and maintenance content, installation and maintenance work order serial number, CRM (Customer Relationship Management) order number, work order type, appointment start time, construction action, construction type, machine building code, activation level, work order description, work order notes and other data information.

[0050] And, the random forest model is a supervised learning model, mainly used for classification and regression problems. Random forest can make predictions by constructing multiple decision trees and aggregating the results of these decision trees to improve the accuracy and robustness of the overall model. It can reduce the risk of overfitting and improve the robustness to outliers and noise by integrating multiple sub-models (decision trees).

[0051] In the present disclosure, the random forest model is used for training, considering that it integrates multiple decision trees, and each tree is independently constructed, which can introduce randomness in the construction process, and when constructing each decision tree, the random forest will randomly select a part of the features from the original features, and randomly extract a part of the samples from the training samples (with replacement sampling). This randomness increases the diversity of the model and helps to avoid overfitting, and the random forest uses the bagging technology, that is, the training data is sampled with replacement to construct each decision tree. This can reduce the model's dependence on specific samples in the training data and improve the generalization ability of the model.

[0052] In step S102, data cleaning and feature extraction are performed on the sample work order data to obtain sample work order features after noise influence is removed.

[0053] Considering that the acquired sample work order data includes multiple data types, the sample work order data corresponding to different data types may be missing, or there may be error interference data affecting the accuracy of the data. Therefore, in some embodiments, the sample work order data can be first classified, and the different types of sample work order data can be cleaned to obtain the sample work order features after removing the influence of error interference.

[0054] Optionally, key feature data can be first selected from the sample work order data. These key feature data can better reflect the corresponding maintenance data of the sample work order. For example, work order data with features such as login time, operation frequency, order amount, IP address change, remuneration amount, etc. can be selected from the sample work order data; then the selected key feature data can be preprocessed, for example, missing value processing, standardization processing, and normalization processing can be performed, so as to remove the influence of error interference and improve the accuracy of the data. Then, feature extraction can be performed on the preprocessed sample work order data.

[0055] In step S103, the sample work order features are divided into multiple sample sets.

[0056] In this step, since the sample work order data is collected from the installation and maintenance business system, the data contains multiple samples, and each sample contains multiple features. Therefore, you can first randomly take a preset proportion of the sample work order features, such as 1 / 10, as the test sample set. For the remaining sample work order features, you can use the 5-fold cross-validation method to train the model, that is, randomly divide the remaining sample work order features into 5 equal parts, and use one of them as the validation set and the remaining 4 as the training set in turn to perform model training.

[0057] In step S104, for each of the sample sets, the importance score of the sample feature cluster to which the sample work order feature belongs is calculated, and the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster is calculated, and the sample work order features are sorted according to the correlation and the importance score of the sample feature cluster to obtain a target sample work order feature sequence.

[0058] The importance score represents the importance of the sample feature cluster to the status identification of the work order; and the relevance represents the relevance between the sample work order features in the sample feature cluster and the classification information.

[0059] In this step, the sample work order features can first be clustered by a clustering algorithm to obtain multiple sample feature clusters, which include multiple sample work order features whose similarities are less than a preset similarity threshold; then the number of out-of-bag error samples corresponding to the sample feature cluster is calculated; then a random sequence transformation is performed on the sample work order features in the sample feature cluster, and the new number of out-of-bag error samples corresponding to the transformed sample feature cluster is calculated; then, based on the number of out-of-bag error samples and the new number of out-of-bag error samples, the importance score corresponding to the sample feature cluster is calculated by a feature importance algorithm; and the correlation between the sample work order features in the sample feature cluster and the classification information is calculated.

[0060] In some embodiments, the sample work order features may be clustered in the following manner to obtain multiple sample feature clusters:

[0061] Method 1: Use KM clustering algorithm (K-Means Clustering Algorithm).

[0062] The KM clustering algorithm measures the similarity between samples based on the distance. The greater the similarity between samples, the smaller the distance between them, and vice versa. First, two parameters need to be determined: the initial number of clusters K and the K initial cluster centers. The algorithm calculates the distance from the samples other than the cluster center to the K cluster centers and assigns it to the cluster with the smallest distance. After clustering the samples, the average value of the samples contained in the K clusters is recalculated and used as the new cluster center of the K clusters, denoted as Repeat the above steps until the sum of squared errors (SSE) of the K clusters converges or reaches the iteration termination condition, at which point clustering ends. The SSE calculation formula is:

[0063]

[0064] Among them, SSE represents the error sum of squares of K clusters, K is the number of clusters, μ i represents the cluster center, and x represents the sample point center.

[0065] Method 2: Using HC algorithm (Hierarchical Clustering Algorithm).

[0066] The HC algorithm builds a hierarchical tree structure by continuously merging and dividing samples. It can be divided into two methods: agglomeration and splitting. Agglomeration uses a bottom-up construction method, treating each sample as a cluster, calculating the similarity between clusters, and merging similar clusters together to form a new cluster until there is only one cluster or the iteration termination condition is reached. Splitting is the opposite of agglomeration. It uses a top-down construction method, treating all samples as one cluster, and each iteration will split into smaller clusters until each sample forms a cluster or the iteration termination condition is reached.

[0067] Method three adopts FCM clustering algorithm (Fuzzy C-Means Clustering Algorithm).

[0068] The FCM clustering algorithm uses membership to constrain the degree to which each sample belongs to a certain cluster, thereby achieving the purpose of sample classification. Assume that the training data set is N*P, and the sample Divide N samples into C categories, where N is the number of samples and P is the number of features of each sample; let U = [u ji ] n×c , where u ji Indicates the degree of membership of the i-th sample X in the data set D to the j-th class. The clustering criterion of the FCM clustering algorithm is to find U and V so that the objective function is minimized. The objective function is expressed as:

[0069]

[0070] Among them, m represents the degree of fuzzification, U and V represent the fuzzy membership matrix and cluster center respectively.

[0071] By adopting the above method, the sample work order features can be clustered, and features with high similarity can be clustered into one cluster.

[0072] In step S105, a preset random forest model is trained according to the target sample work order feature sequence and the sample results annotated for the sample work order to obtain an abnormal work order recognition model.

[0073] Among them, the abnormal work order recognition model is used to determine the recognition result of the work order corresponding to the work order data based on the input work order data, and the recognition result is used to include a normal state and an abnormal state.

[0074] In this step, each sample feature cluster can first be arranged in a first preset order according to the correlation of the sample work order features in the sample feature cluster to obtain an arranged candidate sample work order feature sequence; and then, based on the importance score corresponding to each sample feature cluster, multiple sample feature clusters can be arranged in a second preset order to obtain an arranged target sample work order feature sequence.

[0075] Then, according to the feature sequence of the target sample work order in at least one training set and the sample results of the sample work order annotation in the training set, the preset random forest model is trained using five-fold cross validation to obtain an abnormal work order recognition model.

[0076] The training set includes five equal parts of sample sets, one of which is used as a test sample set, and the remaining four are used as training sample sets.

[0077] Optionally, the target sample work order feature sequence in the test sample set can be input into the random forest model to obtain a sample work order recognition result output by the random forest model; if the sample work order recognition result corresponding to the test sample set is inconsistent with the labeled sample result corresponding to the test sample set, then the remaining four equal parts are used as training sample sets to train the random forest model while adjusting the model parameters of the random forest model until the sample work order recognition result corresponding to the test sample set is consistent with the labeled sample result corresponding to the test sample set.

[0078] And when the sample work order recognition result corresponding to the test sample set is consistent with the labeled sample result corresponding to the test sample set, the trained random forest model is used as the abnormal work order recognition model.

[0079] The above technical solution is adopted to obtain sample work order data of a sample work order and a preset random forest model, wherein the sample work order data is maintenance data information corresponding to a maintenance task performed in a service; data cleaning and feature extraction are performed on the sample work order data to obtain sample work order features after removing the influence of noise; the sample work order features are divided into multiple sample sets; for each of the sample sets, the importance score of the sample feature cluster to which the sample work order feature belongs is calculated, and the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster is calculated, and the sample work order feature is divided into multiple sample sets according to the correlation. and the importance score of the sample feature cluster to obtain the target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the state identification of the work order; the correlation represents the correlation between the sample work order features in the sample feature cluster and the classification information; according to the target sample work order feature sequence and the sample results annotated for the sample work order, the preset random forest model is trained to obtain an abnormal work order recognition model, which is used to determine the recognition result of the work order corresponding to the work order data according to the input work order data, and the recognition result is used to include normal state and abnormal state. In this way, by using the random forest algorithm to analyze and learn the sample work order data of a large number of sample work orders, abnormal work orders can be accurately identified, the detection accuracy and detection efficiency can be improved, and the problem of low detection efficiency caused by relying on physical inspection or manual inspection in traditional detection methods can be avoided.

[0080] In some embodiments, the sample work order data can first be divided into multiple field sets according to the data type of the sample work order data, and the field sets correspond to different data types; then, data cleaning can be performed on each field set to obtain the corresponding field set after removing the influence of noise; and the sample work order data in each field set after removing the noise data is used as the sample work order features after removing the influence of noise.

[0081] Among them, the data type of the sample work order data may include a numerical type, a date type, a category type, and a text type. For example, the numerical type may include work order data such as order amount and remuneration amount; the date type may include work order data such as login time and appointment time; the category type may include work order data such as construction type and business type; the text type may include work order data such as installation and maintenance content, work order description, and work order notes.

[0082] And the field set may include a numeric field set, a date field set, a category field set, and a text field set, and the field set may correspond to the data type of the sample work order data.

[0083] In one possible implementation, when the field set may include a numerical field set, for the numerical field set, the missing values ​​in the numerical field set may be first filled by an exponentially weighted moving average algorithm to obtain a filled numerical field set; then the filled numerical field set may be processed by a standard score normalization processing algorithm to obtain a numerical field set from which noise effects are removed, and the numerical field set from which noise effects are removed may be used as the corresponding field set from which noise effects are removed.

[0084] For example, first, time series data can be established based on the order issuance time corresponding to the sample work order and the sample work order data of the numerical type corresponding to the order issuance time. Then, for each sample work order, the sample work order data of the numerical type corresponding to the sample work order can be obtained. If the data is not obtained, the position corresponding to the order issuance time of the sample work order can be used as the position of the missing value. Then, the EWMA (Exponentially Weighted Moving Average) method can be used to fill in the missing value to maintain the continuity and trend of the time series data.

[0085] Specifically, the following formula can be used to predict missing values:

[0086]

[0087] Among them, EWMA t is the predicted value at time point t, EWMA t-1 The predicted value at time point t-1; Observation t is the observed value at time point t, and λ is a smoothing coefficient adjusted to between 0 and 1.

[0088] Using the EWMA method to fill in values ​​can give more weight to the most recent data points and give exponentially decreasing weights to older data points. This can give more importance to recent data, which is often more relevant to predicting future values. Therefore, by assigning exponentially decreasing weights to older data, changing trends can be captured and predictions for missing values ​​can be adjusted accordingly.

[0089] Then, the z-score (standard score normalization) algorithm can be performed on the time series data filled with EWMA to convert the time series data into sequence data with zero mean and unit variance.

[0090] Specifically, the following formula can be used to perform z-score standardization on the filled time series data:

[0091]

[0092] Where μ is the mean of all EWMA values, σ is the standard deviation, and z is the normalized value at time point t.

[0093] The z-score algorithm is used to standardize the padded time series data, which helps the model to better learn features numerically and improve the model performance to eliminate the impact of different dimensions and standardize the data.

[0094] In another possible implementation, when the field set includes a date field set, the date data in the date field set can be first converted into timestamp information, and then the timestamp information can be normalized to obtain the date field set after removing the influence of noise, and the date field set after removing the influence of noise can be used as the corresponding field set after removing the influence of noise.

[0095] For example, the date data in the date field set can be converted into Unix timestamps (Unix Time, Yonex timestamps, which can facilitate time series analysis and ensure the uniformity of the date data format. Then, the following formula can be used to normalize the Unix timestamp:

[0096]

[0097] Among them, Normalizaed Timestamp indicates the normalized timestamp, Timestamp indicates the timestamp, Min Timestamp indicates the minimum value in the timestamp, and Max Timestamp indicates the maximum value in the timestamp.

[0098] In another possible implementation, when the field set includes a category field set, the preset categories included in the historical work order data can be first obtained, and the preset categories can be filled into the category field set to obtain the filled category field set; then the category data in the category field set can be one-hot encoded, and the encoded category data can be subjected to singular value decomposition processing to obtain the category field set after removing the noise influence, and the category field set after removing the noise influence can be used as the corresponding field set after removing the noise influence.

[0099] Optionally, the preset categories included in the historical work order data can be obtained according to the business logic. Then the preset categories are filled into the category field set to obtain the filled category field set, and then the filled category field set can be converted into onehot encoding, and then singular value decomposition (SVD) is used for dimensionality reduction to obtain a compact feature representation:

[0100] SVD_Feature=USV T ;

[0101] Among them, U, S, and V are the left singular matrix, singular value matrix, and right singular matrix obtained by SVD decomposition, respectively, and SVD_Feature represents the singular value decomposition feature.

[0102] The dimensionality reduction of the data after encoding by singular value decomposition is carried out by taking advantage of the property of SVD that the original matrix can be decomposed into the product of three matrices, namely A = UΣV T , where U and V are orthogonal matrices, Σ is a diagonal matrix containing singular values, and A is the decomposed matrix. In the process of dimensionality reduction, the original matrix can be approximated by retaining the first k largest singular values ​​and their corresponding singular vectors, thereby achieving dimensionality reduction.

[0103] In another possible implementation, when the field set includes a text field set, the importance values ​​of the text data in the text field set can be first obtained through a word frequency-inverse document frequency algorithm; then, a singular value decomposition process can be performed on the importance values ​​of the text data in the text field set to obtain the text field set after removing the influence of noise, and the text field set after removing the influence of noise is used as the corresponding field set after removing the influence of noise.

[0104] The term frequency-inverse document frequency algorithm may include a TF-IDF (Term Frequency-Inverse Document Frequency) algorithm.

[0105] For example, the TF-IDF algorithm may be applied to the text data in the text field set to obtain the importance value of the text data, and SVD may be used to perform dimensionality reduction to obtain a compact feature representation.

[0106] Specifically, the following formula can be used to obtain the importance value of text data:

[0107]

[0108] Among them, IDF is the inverse document frequency, N is the total number of documents, D is the set of all documents, and t is the term.

[0109] In some embodiments, the importance of the sample work order feature can be measured based on the random forest multi-feature permutation, the sample feature obtained after clustering can be permuted, and the importance score of the sample feature can be calculated.

[0110] Optionally, first, the number of decision trees included in the preset random forest model may be obtained, and then the sample work order features may be clustered into i categories to obtain a set of sample feature clusters.

[0111] The number of out-of-bag error samples corresponding to the sample feature cluster can be calculated first, and then all features in the feature cluster of the out-of-bag data can be randomly sequenced while keeping the features in other sample feature clusters unchanged to obtain the out-of-bag data; the number of out-of-bag error samples of the out-of-bag data can then be recalculated, and the above steps can be repeated to finally obtain the new number of out-of-bag error samples corresponding to the transformed sample feature cluster.

[0112] When the number of out-of-bag error samples and the new number of out-of-bag error samples are obtained, the importance score corresponding to the sample feature cluster can be calculated by the following feature importance algorithm formula:

[0113]

[0114] Among them, VI(X Jk ) as the feature cluster X Jk The importance score, N tree is the number of decision trees, and the number of decision trees is the number of new out-of-bag error samples, ErrOOB i is the number of erroneous samples out of the bag.

[0115] Furthermore, each of the sample feature clusters can be arranged in a first preset order according to the relevance of the sample work order features in the sample feature cluster to obtain an arranged candidate sample work order feature sequence; and multiple sample feature clusters can be arranged in a second preset order according to the importance scores corresponding to each of the sample feature clusters to obtain an arranged target sample work order feature sequence.

[0116] Among them, the correlation between the sample work order feature and the classification information in the sample feature cluster is calculated using the Pearson correlation coefficient ρ xy The specific measurement can be calculated by the following formula:

[0117]

[0118] Among them, the ρ xy is the Pearson correlation coefficient, Z x represents the mean of the feature x values, Z y represents the mean of the feature y values, x px Represents the value of feature x, x py Represents the value of feature y.

[0119] Figure 2 is a flow chart of a work order identification method provided in an embodiment of the present application, such as Figure 2 As shown, the method may include the following steps.

[0120] In step S201, target work order data of a target work order and a trained abnormal work order recognition model are obtained.

[0121] The target work order data is maintenance data information corresponding to the maintenance task currently being executed in the service.

[0122] In step S202, data cleaning and feature extraction are performed on the target work order data to obtain target work order features after noise influence is removed.

[0123] In this step, the target work order data received in real time may be subjected to the same preprocessing steps as the sample work order data, including cleaning, standardization, feature extraction, etc.

[0124] For example, work order data with characteristics such as login time, operation frequency, order amount, IP address change, remuneration amount, etc. can be selected from the target work order data; then the selected key feature data can be preprocessed, for example, missing value processing, standardization processing, and normalization processing can be performed, so as to remove the influence of error interference and improve the accuracy of the data, and then the preprocessed target work order data can be subjected to feature extraction.

[0125] In step S203, the target work order feature is input into the abnormal work order recognition model to obtain a recognition result output by the abnormal work order recognition model.

[0126] The recognition result includes a normal state and an abnormal state.

[0127] In this step, the target work order features can be input into the trained abnormal work order recognition model, that is, the random forest model trained by the recognition model training method in the embodiment of the present disclosure is used to calculate the predicted probability of the target work order corresponding to the target work order features, and then the recognition result output by the abnormal work order recognition model is obtained based on the predicted probability.

[0128] Specifically, the predicted probability of the target work order can be calculated in the abnormal work order recognition model by the following formula:

[0129]

[0130] Where P(Fraud|z) is the predicted probability, N is the number of decision trees, I is the indicator function, and Fraud represents anomaly.

[0131] In some embodiments, the target work order features can be predicted by a decision tree in the abnormal work order recognition model to obtain a prediction probability corresponding to the decision tree; the decision tree includes multiple; a weighted average calculation is performed on the multiple prediction probabilities to obtain an average prediction probability after weighted averaging; and the recognition result output by the abnormal work order recognition model is determined based on the average prediction probability.

[0132] Optionally, when the average predicted probability is greater than or equal to a preset probability threshold, the recognition result output by the abnormal work order recognition model is determined to be an abnormal state; when the average predicted probability is less than the preset probability threshold, the recognition result output by the abnormal work order recognition model is determined to be a normal state.

[0133] By adopting the above technical solution, the work order data corresponding to the current target work order can be analyzed and predicted through the trained abnormal work order recognition model, which can deeply integrate data analysis, machine learning and automated monitoring technology to improve the accuracy of work order anomaly detection.

[0134] Figure 3 is a processing flow chart of an identification system provided in an embodiment of the present application, such as Figure 3 As shown, the processing flow of the recognition system may include the following steps:

[0135] In step S301, sample work order data of a sample work order is obtained, and the sample work order data is preprocessed.

[0136] In some embodiments, maintenance data information corresponding to historical maintenance tasks can be comprehensively collected from channels such as online business halls, backend databases, integrated dispatch systems, customer dispatch and customer information systems, etc. The maintenance data information may include login logs, operation records, order details, IP addresses, equipment information, installation and maintenance content, installation and maintenance work order serial numbers, CRM order numbers, work order types, appointment start times, construction actions, construction types, machine building codes, activation levels, work order descriptions, work order notes, and other data information.

[0137] Furthermore, the sample work order data obtained in the above manner may be preprocessed, such as performing missing value processing, standardization processing, and normalization processing, so as to remove the influence of error interference and improve the accuracy of the data. Feature extraction may then be performed on the preprocessed sample work order data.

[0138] In step S302, the pre-processed sample work order data is used to train a preset random forest model.

[0139] In this step, the preset random forest model can be trained using the above-mentioned public recognition model training method, and the specific training method will not be repeated here.

[0140] In step S303, an abnormal work order identification and warning operation is performed, and a response is made to the identification result.

[0141] In this step, the target work order features can be input into the trained abnormal work order recognition model, that is, the random forest model trained by the recognition model training method in the embodiment of the present disclosure is used to calculate the predicted probability of the target work order corresponding to the target work order features, and then the recognition result output by the abnormal work order recognition model is obtained based on the predicted probability.

[0142] Specifically, the predicted probability of the target work order can be calculated in the abnormal work order recognition model by the following formula:

[0143]

[0144] Where P(Fraud|z) is the predicted probability, N is the number of decision trees, I is the indicator function, and Fraud represents anomaly.

[0145] Furthermore, when the predicted probability is greater than or equal to a preset probability threshold, it can be determined that the identification status of the target work order is an abnormal state.

[0146] And when the identification state of the target work order is an abnormal state, the target behavior type of the user corresponding to the target work order is determined according to the prediction category corresponding to the identified prediction probability.

[0147] In step S304, corresponding preventive measures are executed according to the target behavior type of the user.

[0148] For example, the preventive measures may include the following:

[0149] Method 1: Use the account permission management function to limit the order permission of the suspected target account;

[0150] Method 2: Use the verification function to perform secondary verification on these accounts;

[0151] Method 3: For IP addresses that frequently place false orders, use the blacklist management function to add them to the blacklist for interception;

[0152] Method 4: Use system interaction functions to transmit information to integrated dispatching systems, remuneration settlement systems and other frozen settlement functions.

[0153] In step S305, the abnormal work order recognition model is optimized.

[0154] In some embodiments, Figure 4 As shown, the following steps can be used to optimize the abnormal work order recognition model:

[0155] In step S3051, feedback information is collected.

[0156] For example, first, a feedback collection function may be used to systematically collect user feedback information on the recognition model.

[0157] The feedback information includes but is not limited to the recognition results output by each recognition model, especially the false positives and false negatives. A false positive refers to misjudging a normal behavior as a target behavior, while a false negative refers to failing to detect the actual target behavior.

[0158] In step S3052, the parameter type to be adjusted for the trained random forest model is determined based on the feedback information.

[0159] For example, the model fine-tuning function can be used to fine-tune the random forest model based on the collected feedback information. Specifically, by analyzing the cases of false positives and false negatives, the parameter types of the trained random forest model to be adjusted are determined to find out in which cases the model makes incorrect judgments.

[0160] In step S3053, the model parameters are adjusted.

[0161] For example, in response to these problems, the parameters of the model are adjusted to improve the judgment accuracy of the model in similar situations.

[0162] For example, if the model is prone to false positives under certain combinations of features, the weights or thresholds associated with those features can be adjusted to make the model more accurate in identifying the target behavior.

[0163] In step S3054, the trained random forest model is retrained.

[0164] For example, the model is retrained with new data regularly to adapt it to the changing target behavior patterns. Over time, new patterns or trends may emerge in the target behavior, so the model needs to be updated in time to maintain its effectiveness.

[0165] In the retraining process, the random forest algorithm is trained using the training function. First, prepare new data that covers the latest user behavior patterns and possible target behavior types.

[0166] Specifically, this data can be input into the random forest algorithm to let the algorithm relearn and adjust the parameters of the model. During the training process, the algorithm will automatically optimize the structure and parameters of the model according to the features and labels in the data to improve the prediction ability of the model. Until the preset end condition is reached, such as the performance of the model reaches a certain accuracy rate on the validation set or stabilizes at a good level, a new abnormal work order recognition model is obtained.

[0167] The above technical solution can automatically adjust the parameters of the random forest algorithm, optimize the model structure, and even introduce new feature variables based on the user's feedback on the recognition results after each recognition, so as to continuously improve the accuracy of target detection and the pertinence of prevention strategies. This closed-loop feedback mechanism ensures that the system can keep up with the development of the target's methods and always maintain efficient protection capabilities.

[0168] Figure 5 Schematic diagram of the execution steps of building, training and applying a recognition model provided in an embodiment of the present application, such as Figure 5 As shown, the following steps may be included:

[0169] In step S401, the training algorithm hyperparameters of the random forest algorithm are initialized.

[0170] For example, basic parameters of the random forest model, including the number of decision trees, maximum depth, minimum number of samples, etc., may be set to prepare for model training.

[0171] And determine the exponential weighted moving average (EWMA) time weight λ, for example, the algorithm hyperparameter setting can be set to λ=0.5 according to an empirical value, and the dimension k of the singular value decomposition dimensionality reduction can be set to k=30 according to an empirical value.

[0172] In step S402, weighted bootstrap sampling is performed on the training data set.

[0173] Optionally, to deal with the class imbalance problem, weighted bootstrap sampling can be performed on the training data set, which can increase the weight of positive samples and reduce the model's bias towards the majority class. Specifically, the weight calculation formula is:

[0174] Weighti=n×I(yi=minority class)+1×(1-I(yi=

[0175] majority class);

[0176] Among them, minority class is the minority class sample, n is the weight, and n>1, majority class is the majority class sample.

[0177] You can also assign weights directly in the code according to the sample category and the specific troubleshooting of the business scenario, for example, for minority class samples: Weighti=n; for majority class samples: Weighti=1.

[0178] In step S403, target work order data of the target work order is acquired, and the target work order data is preprocessed.

[0179] In this step, the data type corresponding to the target work order data can be first determined, wherein the data type can include a numerical type, a date type, a category type, and a text type. For example, the numerical type can include work order data such as order amount and remuneration amount; the date type can include work order data such as login time and appointment time; the category type can include work order data such as construction type and business type; the text type can include work order data such as installation and maintenance content, work order description, and work order notes.

[0180] In some embodiments, when the data type corresponding to the target work order data is a numeric type, considering that there is no missing data if the data type is a numeric type, the filled numeric field set can be directly processed by the standard score normalization processing algorithm to obtain the target work order data with noise removed.

[0181] For example, a time series can be first established based on the order issuance time corresponding to the target work order and the target work order data of a numerical type corresponding to the order issuance time; then a z-score (standard score normalization) algorithm can be executed on the time series to convert the time series data into sequence data with zero mean and unit variance.

[0182] Specifically, the following formula can be used to perform z-score normalization on the filled time series:

[0183]

[0184] Among them, μ is the value of the target work order data, σ is the standard deviation, and z is the standardized value at time point t.

[0185] The z-score algorithm is used to standardize the padded time series, which helps the model to better learn features numerically and improve the model performance to eliminate the impact of different dimensions and standardize the data.

[0186] In other embodiments, when the data type corresponding to the target work order data is a date type, the date data may be converted into timestamp information, and then the timestamp information may be normalized to obtain the target work order data after noise influence is removed.

[0187] For example, the date data in the date field set can be converted into Unix timestamps (Unix Time, Yonex timestamps, which can facilitate time series analysis and ensure the uniformity of the date data format, and then the Unix timestamps are normalized using the following formula:

[0188]

[0189] Among them, Normalizaed Timestamp indicates the normalized timestamp, Timestamp indicates the timestamp, Min Timestamp indicates the minimum value in the timestamp, and Max Timestamp indicates the maximum value in the timestamp.

[0190] In other embodiments, when the data type corresponding to the target work order data is a category type, the target work order data can be one-hot encoded, and the encoded category data can be subjected to singular value decomposition to obtain the target work order data after noise influence is removed.

[0191] And in other embodiments, when the data type corresponding to the target work order data is a text type, the important values ​​corresponding to the target work order data can be first obtained through the word frequency-inverse document frequency algorithm, and then the important values ​​can be subjected to singular value decomposition processing to obtain the target work order data after removing the influence of noise.

[0192] Optionally, the target work order data after the noise data is removed can be used as the target work order features after the noise influence is removed.

[0193] In step S404, the preprocessed target work order data is input into the abnormal work order recognition model.

[0194] In step S405, prediction is performed through each decision tree, and then the prediction results of all trees are integrated to determine the final classification.

[0195] The prediction formula is:

[0196] mode(x) = mode{Tree i (x)|i=1,2,…,N};

[0197] Where mode is the mode function, Tree i (x) is the prediction of the i-th tree for sample x.

[0198] In step S406, the probability value is output.

[0199] Among them, the final prediction probability is the weighted average of the prediction probabilities of all decision trees.

[0200] In step S407, the target detection threshold is optimized.

[0201] For example, based on the performance of the model on the validation set, the optimal target detection threshold is determined using the area under the ROC curve (AUC), and the further steps include:

[0202] Step A1: Define performance indicators.

[0203] True Positive Rate (TPR):

[0204] False Positive Rate (FPR):

[0205] Among them, TP, FN, FP, and TN represent the number of true positive examples, false negative examples, false positive examples, and true negative examples, respectively, and these numbers are the prediction results based on the given threshold θ.

[0206] Step A2, drawing a ROC curve (Receiver Operating Characteristic Curve): using different values ​​of θ, calculating the corresponding TPR(θ) and FPR(θ), and plotting these points on the coordinate axis to form a ROC curve.

[0207] Step A3: Calculate the AUC (Area Under Curve) and use the AUC to determine the optimal anomaly detection threshold.

[0208] Among them, the area under the ROC curve AUC can be calculated by numerical methods (such as trapezoidal rule). This step is mainly used to evaluate the overall performance of the model.

[0209] For example, if we want to minimize the false positive rate while maintaining a high recall rate, we can define a cost function, such as the F1 score:

[0210] F1 score:

[0211] Among them, precision (Precision) and recall rate (Recall, namely TPR) are functions of threshold θ:

[0212] Accuracy

[0213] The recall rate (Recall) has been defined as TPR before;

[0214] Then, we can choose a threshold θ that maximizes the F1 score. * As the optimal threshold. This usually involves a search process, such as traversing all possible thresholds, calculating the F1 score under each threshold, and finding the threshold corresponding to the maximum value.

[0215] In step S408, real-time monitoring and early warning are performed on the potential target behaviors corresponding to the detected abnormal work orders.

[0216] Specifically, when the predicted probability exceeds a dynamically adjusted threshold, an alarm may be triggered and corresponding measures may be taken.

[0217] In step S409, a cross-validation method is used to evaluate the model performance and adjust the model parameters.

[0218] By adopting the above technical solution, in the process of target behavior detection, dynamic thresholds and threshold judgment mechanisms can be used to automatically adjust thresholds according to historical data and real-time feedback to adapt to changes in target behavior patterns. At the same time, when the predicted probability exceeds the set threshold, the system automatically triggers an alarm, thereby realizing real-time monitoring and rapid response of target behavior detection. Real-time target behavior detection facilitates instant analysis of newly generated user behavior data and uses trained models for prediction, thereby improving the system's adaptability and ensuring that the detection mechanism can maintain high accuracy under different circumstances.

[0219] In some embodiments, Figure 6 As shown, the target sample work order feature sequence can be obtained through the following steps:

[0220] In step S1041, the sample work order features are clustered to cluster features with high similarity into one cluster.

[0221] In step S1042, the feature clusters obtained after clustering are permuted.

[0222] In step S1043, the number of out-of-bag error samples of the out-of-bag data corresponding to the decision tree is calculated, and the sample work order features in the sample feature cluster are subjected to random sequence transformation, and the number of new out-of-bag error samples corresponding to the transformed sample feature cluster is calculated.

[0223] In step S1044, the importance score corresponding to the sample feature cluster is calculated using a feature importance algorithm.

[0224] In step S1045, the sample work order features are sorted according to the relevance and the importance score of the sample feature cluster to obtain a target sample work order feature sequence.

[0225] By adopting the above technical solution, the feature selection algorithm of random forest multi-feature replacement can be used to perform deep learning and pattern mining on the pre-processed user behavior data, and a high-precision target recognition model can be constructed. The core of this module uses the feature selection algorithm of random forest multi-feature replacement to perform deep learning and pattern mining on the pre-processed user behavior data.

[0226] In some embodiments, Figure 7 As shown, the following steps can be used to clean the sample work order data and extract its features to obtain the sample work order features after removing the noise:

[0227] In step S1021, key features are extracted.

[0228] Optionally, key feature data can be first selected from the sample work order data. These key feature data can better reflect the corresponding maintenance data of the sample work order. For example, work order data about login time, operation frequency, order amount, IP address change, remuneration amount and other features can be selected from the sample work order data.

[0229] In step S1022, the sample work order data is divided into a plurality of field sets according to the data type of the sample work order data.

[0230] Among them, the field set corresponds to different data types.

[0231] When the field set includes a numerical field set, steps S1023 and S1024 are executed;

[0232] When the field set includes a date field set, execute steps S1025 and S1026;

[0233] In the case where the field set includes a category field set, steps S1027 and S1028 are executed;

[0234] In the case where the field set includes a text field set, steps S1029 and S10210 are executed;

[0235] In step S1023, the missing values ​​in the numerical field set are filled by using an exponentially weighted moving average algorithm to obtain a filled numerical field set.

[0236] In step S1024, the filled numerical field set is processed by a standard score normalization processing algorithm to obtain a numerical field set after removing the noise influence, and the numerical field set after removing the noise influence is used as the corresponding field set after removing the noise influence.

[0237] In step S1025, the date data in the date field set is converted into timestamp information.

[0238] In step S1026, the timestamp information is normalized to obtain the date field set after the noise effect is removed, and the date field set after the noise effect is removed is used as the corresponding field set after the noise effect is removed.

[0239] In step S1027, the preset category included in the historical work order data is obtained, and the preset category is filled into the category field set to obtain a filled category field set.

[0240] In step S1028, the category data in the category field set is one-hot encoded, and the encoded category data is subjected to singular value decomposition to obtain the category field set after removing the influence of noise, and the category field set after removing the influence of noise is used as the corresponding field set after removing the influence of noise.

[0241] In step S1029, the importance value of the text data in the text field set is obtained by using a word frequency-inverse document frequency algorithm.

[0242] In step S10210, singular value decomposition is performed on the important values ​​of the text data in the text field set to obtain the text field set after removing the influence of noise, and the text field set after removing the influence of noise is used as the corresponding field set after removing the influence of noise.

[0243] By adopting the above technical solution, various new features based on business experience are introduced by using the mean to fill missing values, z-score standardization, and specific field calculations based on business logic, which helps to capture the patterns of target behaviors, improves the model's ability to recognize target behaviors, significantly improves the consistency and quality of data, and provides more reliable input for the model, thereby improving the model's predictive accuracy. Specific processing of different types of fields is also introduced, which provides an additional dimension for the model, helps to capture the subtle differences in target behaviors, and can enhance the stability and predictive ability of the data.

[0244] Figure 8 is a block diagram of a recognition model training device provided in an embodiment of the present application, such as Figure 8 As shown, the device 500 may include the following modules.

[0245] A first acquisition module 501 is used to acquire sample work order data of a sample work order and a preset random forest model, wherein the sample work order data is maintenance data information corresponding to a maintenance task performed in the service;

[0246] The first cleaning module 502 is used to perform data cleaning and feature extraction on the sample work order data to obtain sample work order features after removing noise influence;

[0247] A division module 503 is used to divide the sample work order feature into multiple sample sets;

[0248] The determination module 504 is used to calculate, for each of the sample sets, the importance score of the sample feature cluster to which the sample work order feature belongs, and calculate the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster, and sort the sample work order features according to the correlation and the importance score of the sample feature cluster to obtain a target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the state identification of the work order; the correlation represents the correlation between the sample work order feature in the sample feature cluster and the classification information;

[0249] The training module 505 is used to train the preset random forest model according to the feature sequence of the target sample work order and the sample results annotated for the sample work order to obtain an abnormal work order recognition model. The abnormal work order recognition model is used to determine the recognition result of the work order corresponding to the work order data based on the input work order data. The recognition result is used to include a normal state and an abnormal state.

[0250] Optionally, the first cleaning module 502 is used to divide the sample work order data into a plurality of field sets according to the data type of the sample work order data, and the field sets correspond to different data types;

[0251] Perform data cleaning on each field set to obtain the corresponding field set after removing the noise influence;

[0252] The sample work order data in each field set after the noise data is removed is used as the sample work order features after the noise effect is removed.

[0253] Optionally, the field set includes a numerical field set; the first cleaning module 502 is used to fill in missing values ​​in the numerical field set by an exponentially weighted moving average algorithm to obtain a filled numerical field set;

[0254] The filled numerical field set is processed by a standard score normalization processing algorithm to obtain a numerical field set after noise influence is removed, and the numerical field set after noise influence is removed is used as the corresponding field set after noise influence is removed.

[0255] Optionally, the field set includes a date field set; the first cleaning module 502 is used to convert date data in the date field set into timestamp information;

[0256] The timestamp information is normalized to obtain the date field set after the noise effect is removed, and the date field set after the noise effect is removed is used as the corresponding field set after the noise effect is removed.

[0257] Optionally, the field set includes a category field set; the first cleaning module 502 is used to obtain a preset category included in the historical work order data, and fill the preset category into the category field set to obtain a filled category field set;

[0258] The category data in the category field set is one-hot encoded, and the encoded category data is subjected to singular value decomposition to obtain the category field set after removing the influence of noise, and the category field set after removing the influence of noise is used as the corresponding field set after removing the influence of noise.

[0259] Optionally, the field set includes a text field set; the first cleaning module 502 is used to obtain the importance value of the text data in the text field set by using a word frequency-inverse document frequency algorithm;

[0260] The important values ​​of the text data in the text field set are subjected to singular value decomposition processing to obtain the text field set after the noise influence is removed, and the text field set after the noise influence is removed is used as the corresponding field set after the noise influence is removed.

[0261] Optionally, the determination module 504 is used to cluster the sample work order features by a clustering algorithm to obtain a plurality of sample feature clusters, wherein the sample feature clusters include a plurality of sample work order features whose similarity is less than a preset similarity threshold;

[0262] Calculate the number of out-of-bag error samples corresponding to the sample feature cluster;

[0263] Performing a random sequence transformation on the sample work order feature in the sample feature cluster, and calculating the number of new out-of-bag error samples corresponding to the transformed sample feature cluster;

[0264] According to the number of out-of-bag error samples and the new number of out-of-bag error samples, the importance score corresponding to the sample feature cluster is calculated by a feature importance algorithm;

[0265] Calculate the correlation between the sample work order feature and the classification information in the sample feature cluster.

[0266] Optionally, the determination module 504 is used to arrange each sample feature cluster according to the relevance of the sample work order features in the sample feature cluster in a first preset order to obtain an arranged candidate sample work order feature sequence;

[0267] According to the importance score corresponding to each of the sample feature clusters, the plurality of sample feature clusters are arranged in a second preset order to obtain an arranged feature sequence of the target sample work order.

[0268] Optionally, the training module 505 is used to train a preset random forest model using five-fold cross validation based on the target sample work order feature sequence in at least one training set and the sample results of the sample work order annotation in the training set to obtain an abnormal work order recognition model.

[0269] Optionally, the training set includes a sample set of five equal parts, wherein one equal part is used as a test sample set, and the remaining four equal parts are used as a training sample set; the training module 505 is used to train the preset random forest model using five-fold cross validation according to the feature sequence of the target sample work order in at least one training set and the sample results of the sample work order annotation in the training set, and obtain the abnormal work order recognition model including:

[0270] Inputting the target sample work order feature sequence in the test sample set into the random forest model to obtain the sample work order recognition result output by the random forest model;

[0271] If the sample work order recognition result corresponding to the test sample set is inconsistent with the labeled sample result corresponding to the test sample set, then the random forest model is trained using the remaining four equal parts as the training sample set while adjusting the model parameters of the random forest model until the sample work order recognition result corresponding to the test sample set is consistent with the labeled sample result corresponding to the test sample set;

[0272] When the sample work order recognition result corresponding to the test sample set is consistent with the labeled sample result corresponding to the test sample set, the trained random forest model is used as the abnormal work order recognition model.

[0273] By adopting the above-mentioned device, it is possible to obtain sample work order data of a sample work order and a preset random forest model, wherein the sample work order data is maintenance data information corresponding to a maintenance task performed in a service; perform data cleaning and feature extraction on the sample work order data to obtain sample work order features after removing the influence of noise; divide the sample work order features into multiple sample sets; for each of the sample sets, calculate the importance score of the sample feature cluster to which the sample work order feature belongs, and calculate the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster, and divide the sample work order features into multiple sample sets according to the correlation. and the importance score of the sample feature cluster to obtain the target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the state identification of the work order; the correlation represents the correlation between the sample work order features in the sample feature cluster and the classification information; according to the target sample work order feature sequence and the sample results annotated for the sample work order, the preset random forest model is trained to obtain an abnormal work order recognition model, which is used to determine the recognition result of the work order corresponding to the work order data according to the input work order data, and the recognition result is used to include normal state and abnormal state. In this way, by using the random forest algorithm to analyze and learn the sample work order data of a large number of sample work orders, abnormal work orders can be accurately identified, the detection accuracy and detection efficiency can be improved, and the problem of low detection efficiency caused by relying on physical inspection or manual inspection in traditional detection methods can be avoided.

[0274] Fig. 9 is a block diagram of an identification device provided in an embodiment of the present application, such as Fig. 9 As shown, the device 600 may include the following modules.

[0275] The second acquisition module 601 is used to acquire target work order data of a target work order and a trained abnormal work order recognition model, wherein the target work order data is maintenance data information corresponding to a maintenance task currently being executed in the service;

[0276] The second cleaning module 602 is used to perform data cleaning and feature extraction on the target work order data to obtain the target work order features after removing the noise influence;

[0277] The recognition module 603 is used to input the target work order feature into the abnormal work order recognition model to obtain the recognition result output by the abnormal work order recognition model, and the recognition result is used to include a normal state and an abnormal state.

[0278] Optionally, the second cleaning module 602 is used to perform data cleaning on the target work order data according to the data type of the target work order data to obtain target work order features after noise influence is removed.

[0279] Optionally, the identification module 603 is used to predict the target work order feature through a decision tree in the abnormal work order identification model to obtain a prediction probability corresponding to the decision tree; the decision tree includes multiple;

[0280] Performing weighted average calculation on multiple prediction probabilities to obtain an average prediction probability after weighted average;

[0281] According to the average predicted probability, the recognition result output by the abnormal work order recognition model is determined.

[0282] Optionally, the identification module 603 is used to determine that the identification result output by the abnormal work order identification model is an abnormal state when the average predicted probability is greater than or equal to a preset probability threshold;

[0283] When the average predicted probability is less than the preset probability threshold, it is determined that the recognition result output by the abnormal work order recognition model is in a normal state.

[0284] By using the above device, the work order data corresponding to the current target work order can be analyzed and predicted through the trained abnormal work order recognition model, which can deeply integrate data analysis, machine learning and automated monitoring technology to improve the accuracy of work order anomaly detection.

[0285] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0286] The present invention also provides an electronic device, see Fig.10 , including: a processor 701, a memory 702, and a computer program 7021 stored in the memory and executable on the processor, wherein when the processor executes the program, the recognition model training device and the recognition device of the aforementioned embodiment are implemented.

[0287] The present invention also provides a readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the recognition model training device and recognition device of the aforementioned embodiment.

[0288] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0289] It should be noted that the various information and data obtained in the embodiments of the present invention are all obtained with the authorization of the information / data holder.

[0290] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming voice. It should be understood that various programming voices can be utilized to realize the content of the present invention described herein, and the description of the above specific voices is for the purpose of disclosing the best mode of the present invention.

[0291] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0292] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0293] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0294] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention may also be implemented as a device or apparatus program for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0295] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0296] The user information (including but not limited to the user's device information, user personal information, etc.) and related data involved in the present invention are all information authorized by the user or by all parties.

[0297] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0298] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0299] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A recognition model training method, characterized in that: The method comprises: Obtaining sample work order data of a sample work order and a preset random forest model, wherein the sample work order data is maintenance data information corresponding to a maintenance task performed in the service; Performing data cleaning and feature extraction on the sample work order data to obtain sample work order features after removing noise influence; Dividing the sample work order features into multiple sample sets; For each of the sample sets, the importance score of the sample feature cluster to which the sample work order feature belongs is calculated, and the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster is calculated, and the sample work order features are sorted according to the correlation and the importance score of the sample feature cluster to obtain a target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the status identification of the work order; the correlation represents the correlation between the sample work order feature in the sample feature cluster and the classification information; According to the target sample work order feature sequence and the sample results of the sample work order annotation, the preset random forest model is trained to obtain an abnormal work order recognition model. The abnormal work order recognition model is used to determine the recognition result of the work order corresponding to the work order data based on the input work order data. The recognition result is used to include a normal state and an abnormal state.

2. The method according to claim 1, characterized in that The cleaning and feature extraction of the sample work order data to obtain the sample work order features after removing the noise influence includes: According to the data type of the sample work order data, the sample work order data is divided into a plurality of field sets, wherein the field sets correspond to different data types; Perform data cleaning on each field set to obtain the corresponding field set after removing the noise influence; The sample work order data in each field set after the noise data is removed is used as the sample work order features after the noise influence is removed.

3. The method according to claim 2, characterized in that The field set includes a numerical field set; the data cleaning of each field set to obtain a corresponding field set after removing the noise influence includes: Filling the missing values ​​in the numerical field set by using an exponentially weighted moving average algorithm to obtain a filled numerical field set; The filled numerical field set is processed by a standard score normalization processing algorithm to obtain a numerical field set after removing the noise influence, and the numerical field set after removing the noise influence is used as the corresponding field set after removing the noise influence.

4. The method according to claim 2, characterized in that: The field set includes a date field set; the data cleaning of each field set to obtain a corresponding field set after removing the noise influence includes: Convert the date data in the date field set into timestamp information; The timestamp information is normalized to obtain the date field set after noise influence is removed, and the date field set after noise influence is removed is used as the corresponding field set after noise influence is removed.

5. The method according to claim 2, characterized in that: The field set includes a category field set; the data cleaning of each field set to obtain a corresponding field set after removing the noise influence includes: Acquire preset categories included in the historical work order data, and fill the preset categories into the category field set to obtain a filled category field set; The category data in the category field set are one-hot encoded, and the encoded category data are subjected to singular value decomposition processing to obtain the category field set after removing the influence of noise, and the category field set after removing the influence of noise is used as the corresponding field set after removing the influence of noise.

6. The method according to claim 2, characterized in that The field set includes a text field set; the data cleaning of each field set to obtain a corresponding field set after removing the noise influence includes: Obtaining the importance value of the text data in the text field set by using a word frequency-inverse document frequency algorithm; The important values ​​of the text data in the text field set are subjected to singular value decomposition processing to obtain the text field set after the noise influence is removed, and the text field set after the noise influence is removed is used as the corresponding field set after the noise influence is removed.

7. The method according to claim 1, characterized in that The calculating the importance score of the sample feature cluster to which the sample work order feature belongs, and calculating the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster include: Clustering the sample work order features by a clustering algorithm to obtain a plurality of sample feature clusters, wherein the sample feature clusters include a plurality of sample work order features whose similarity is less than a preset similarity threshold; Calculate the number of out-of-bag error samples corresponding to the sample feature cluster; Performing random sequence transformation on the sample work order features in the sample feature cluster, and calculating the number of new out-of-bag error samples corresponding to the transformed sample feature cluster; Calculate the importance score corresponding to the sample feature cluster by a feature importance algorithm according to the number of out-of-bag error samples and the new number of out-of-bag error samples; The correlation between the sample work order features and the classification information in the sample feature cluster is calculated.

8. The method according to claim 1, characterized in that The step of sorting the sample work order features according to the relevance and the importance score of the sample feature cluster to obtain a target sample work order feature sequence includes: For each of the sample feature clusters, the sample work order features in the sample feature cluster are arranged in a first preset order according to the relevance thereof, so as to obtain an arranged candidate sample work order feature sequence; According to the importance score corresponding to each of the sample feature clusters, the plurality of sample feature clusters are arranged in a second preset order to obtain the arranged target sample work order feature sequence.

9. The method according to claim 1, characterized in that: The step of training a preset random forest model according to the target sample work order feature sequence and the sample results of labeling the sample work orders to obtain an abnormal work order recognition model includes: According to the target sample work order feature sequence in at least one training set and the sample results of the sample work order annotation in the training set, the preset random forest model is trained using five-fold cross validation to obtain an abnormal work order recognition model.

10. The method according to claim 9, characterized in that The training set includes five equal parts of sample sets, wherein one equal part is used as a test sample set, and the remaining four equal parts are used as training sample sets; The method of training a preset random forest model using five-fold cross validation based on the target sample work order feature sequence in at least one training set and the sample results of the sample work order annotation in the training set to obtain an abnormal work order recognition model includes: Inputting the target sample work order feature sequence in the test sample set into the random forest model to obtain the sample work order recognition result output by the random forest model; If the sample work order recognition result corresponding to the test sample set is inconsistent with the labeled sample result corresponding to the test sample set, then the random forest model is trained using the remaining four equal parts as training sample sets while adjusting the model parameters of the random forest model until the sample work order recognition result corresponding to the test sample set is consistent with the labeled sample result corresponding to the test sample set; When the sample work order recognition result corresponding to the test sample set is consistent with the labeled sample result corresponding to the test sample set, the trained random forest model is used as the abnormal work order recognition model.

11. A work order identification method, characterized in that: The method comprises: Obtain target work order data of a target work order and a trained abnormal work order recognition model, wherein the target work order data is maintenance data information corresponding to a maintenance task currently being performed in the service; Performing data cleaning and feature extraction on the target work order data to obtain target work order features after removing noise influence; The target work order feature is input into the abnormal work order recognition model to obtain a recognition result output by the abnormal work order recognition model, wherein the recognition result includes a normal state and an abnormal state.

12. A recognition model training device, characterized in that: The device comprises: A first acquisition module is used to acquire sample work order data of a sample work order and a preset random forest model, wherein the sample work order data is maintenance data information corresponding to a maintenance task performed in the service; A first cleaning module is used to perform data cleaning and feature extraction on the sample work order data to obtain sample work order features after noise influence is removed; A division module, used for dividing the sample work order features into multiple sample sets; A determination module is used to calculate, for each of the sample sets, an importance score of the sample feature cluster to which the sample work order feature belongs, and calculate the correlation between the sample work order feature and other sample work order features in the corresponding sample feature cluster, and sort the sample work order features according to the correlation and the importance score of the sample feature cluster to obtain a target sample work order feature sequence; the importance score represents the importance of the sample feature cluster to the status identification of the work order; the correlation represents the correlation between the sample work order feature in the sample feature cluster and the classification information; A training module is used to train a preset random forest model according to the target sample work order feature sequence and the sample results of the sample work order annotation to obtain an abnormal work order recognition model. The abnormal work order recognition model is used to determine the recognition result of the work order corresponding to the work order data based on the input work order data, and the recognition result is used to include a normal state and an abnormal state.

13. An identification device, characterized in that: The device comprises: A second acquisition module is used to acquire target work order data of a target work order and a trained abnormal work order recognition model, wherein the target work order data is maintenance data information corresponding to a maintenance task currently being executed in the service; A second cleaning module is used to perform data cleaning and feature extraction on the target work order data to obtain target work order features after removing noise influence; The recognition module is used to input the target work order feature into the abnormal work order recognition model to obtain the recognition result output by the abnormal work order recognition model, and the recognition result is used to include a normal state and an abnormal state.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 11 when executed by the processor.

Images