Automatic classification method and device for house quality safety complaints

Abstract

The invention discloses a house quality safety complaint automatic classification method and device, and the method comprises the steps: obtaining house quality safety complaint data, and converting the data into table data in a unified manner; performing feature extraction and semantic enhancement on the table data, and constructing a multi-task learning model combined with a dynamic adaptive attention mechanism; training, verifying and optimizing the multi-task learning model; performing model evaluation and verification on the trained multi-task learning model; and deploying the trained multi-task learning model into an actual complaint processing system, receiving complaint data, realizing automatic classification and real-time severity scoring, and continuously optimizing the system based on a feedback mechanism. According to the method, the classification accuracy of complex and multi-theme complaint texts is remarkably improved; unified optimization and integration of structured data and unstructured data are realized, and high-quality input features are provided for the model.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent housing management, and in particular to a method and device for automatically classifying housing quality and safety complaints. Background Art

[0002] With the acceleration of urbanization and the improvement of residents' living standards, housing quality and safety issues are increasingly attracting attention from all walks of life. The handling of owners' complaints about housing quality issues has become an indispensable part of property management. However, traditional complaint handling methods mostly rely on manual answering, recording, classification and feedback. This method is not only inefficient, but also easily affected by human factors, resulting in unsatisfactory handling results. Therefore, exploring an efficient, intelligent and accurate complaint handling method has become an urgent need in the industry.

[0003] Therefore, there are some automatic complaint content recognition solutions based on neural network models in the industry to achieve automatic complaint processing. However, these existing technical solutions have the following disadvantages:

[0004] (1) Poor domain adaptability: Existing solutions often lack in-depth research and customized optimization in the field of housing quality and safety complaints, resulting in poor results in handling complaints in specific areas;

[0005] (2) Limited recognition accuracy: Since housing quality and safety complaints involve many professional terms and specific scenarios, existing speech recognition and natural language processing technologies often have difficulty achieving high accuracy when dealing with these complex situations;

[0006] (3) Complicated processing flow: Most existing systems still require manual intervention to reconfirm and process the recognition results, which makes the entire processing flow still cumbersome and time-consuming;

[0007] (4) Data security issues: When processing complaint data, existing systems may face security issues such as data leakage and privacy protection, thus posing potential risks to users and enterprises.

[0008] Therefore, although the existing implementation scheme has improved the efficiency of complaint handling to a certain extent, it still has obvious deficiencies in terms of domain adaptability, recognition accuracy, processing flow and data security, which leads to inaccurate classification and inaccurate severity rating when processing category imbalance and multi-topic texts, and is not suitable for handling complex and multi-topic complaint texts. In view of this, it is necessary to provide a method and device for automatic classification of housing quality and safety complaints to overcome the above defects. Summary of the invention

[0009] The purpose of the present invention is to provide a method and device for automatic classification of housing quality and safety complaints, aiming to solve the problems of inaccurate classification and inaccurate severity scoring in existing similar methods when dealing with category imbalance and multi-topic texts, and to improve the model's understanding and classification accuracy of complex, multi-topic complaint texts.

[0010] In order to achieve the above-mentioned object, the first aspect of the present invention provides a method for automatically classifying housing quality and safety complaints, comprising:

[0011] Step S10: Obtain housing quality and safety complaint data from the property management system, online complaint platform, emails and telephone records, and use a preset multimodal neural network to uniformly convert the housing quality and safety complaint data into tabular data; wherein the tabular data includes complaint ID, complaint type, complaint time and location, complaint content summary, processing time, complaint severity score and processing status;

[0012] Step S20: performing feature extraction and semantic enhancement on the table data, and constructing a multi-task learning model combined with a dynamic adaptive attention mechanism; wherein the multi-task learning model includes a shared encoder and a classification task output head and a regression task output head; the classification task output head is used to map the hidden layer representation processed by the encoder to the probability distribution of various types of complaints, so as to be responsible for determining the specific category of the complaint; the regression task output head is used to map the hidden layer representation processed by the encoder to the output value of the regression task, so as to predict the severity or urgency score of the complaint;

[0013] Step S30: obtaining a data set required for training the multi-task learning model, dividing the data set into a training set, a validation set and a test set, and then training, validating and optimizing the multi-task learning model;

[0014] Step S40: performing model evaluation and verification on the trained multi-task learning model, and further tuning the parameters of the model that does not meet the requirements according to the evaluation results, so as to achieve balanced performance of the model in all categories;

[0015] Step S50: deploy the trained multi-task learning model to the actual complaint handling system, receive complaint data, implement automatic classification and real-time severity scoring, and continuously optimize the system based on a feedback mechanism.

[0016] In a preferred embodiment, the multimodal neural network includes a structured data processing subnetwork, a text processing subnetwork, a speech processing subnetwork and a feature fusion layer;

[0017] Wherein, the step S10 includes:

[0018] Step S11: performing structured data processing on the complaint data of the property management system, and inputting the obtained structured data into the structured data processing subnetwork;

[0019] Step S12: performing unstructured data processing on the complaint data of the online complaint platform and the email, and inputting the word embedding vector and / or sentence embedding vector generated by the processing into the text processing sub-network;

[0020] Step S13: performing voice data processing on the complaint data recorded in the telephone record to generate text output, converting the generated text data into a word embedding vector, and then inputting the generated word embedding vector into the voice processing subnetwork;

[0021] Step S14: Integrate the features from the structured data, text data and voice data through the feature fusion layer to form a unified feature representation based on a table.

[0022] In a preferred embodiment, the structured data processing subnetwork is composed of a series of fully connected layers, and performs feature extraction and processing on the structured data according to a first preset formula; the first preset formula is as follows:

[0023] h1=ReLU(W1x+b1),h2=ReLU(W2h1+b2),y structured =W3h2+b3,

[0024] Where x is the input structured data vector, W1, W2, W3 are weight matrices, b1, b2, b3 are biases, ReLU is an activation function used to introduce nonlinearity, and y structured is the output feature of the structured data processing subnetwork;

[0025] The text processing sub-network is constructed based on the Transformer architecture and performs semantic understanding and feature extraction on the text according to the second preset formula; the second preset formula is as follows:

[0026] In the formula, K represents the key matrix, which is extracted from the input word embedding or the output of the previous layer and is used to represent the features that need to be compared; Q represents the query matrix, which is also extracted from the input word embedding or the output of the previous layer and is used to find features related to the key matrix K; the key matrix K measures the similarity by performing a dot product with the query matrix Q; V represents the value matrix, which corresponds to the actual content of the input and generates the final output through weighted combination; Attention represents the output of the attention mechanism and is used to perform weighted averaging on the value matrix V; softmax represents a normalization function that converts the dot product result of the query matrix Q and the key matrix K into a probability distribution so that the sum of the attention weights is 1; is the scaled dot product of the query matrix Q and the key matrix K, d k is the dimension of the key vector, the scaling factor Used to avoid gradient vanishing due to large values; T is the matrix transpose;

[0027] The speech processing subnetwork is used to first convert the speech data into text, and then perform corresponding data processing on the text through the second preset formula.

[0028] In a preferred embodiment, the step S11 includes:

[0029] Step S111: extracting structured data containing detailed records of housing quality complaints from the property management system, including complaint ID, complaint type, complaint time and location, problem description, and processing status;

[0030] Step S112: identifying and processing missing values ​​and outliers on the extracted data;

[0031] Step S113: Encode the categorical data to obtain categorical features;

[0032] Step S114: split the timestamp data into multiple time fields including “year, month, day, hour, minute”, and perform periodic feature processing to capture the periodic features of time;

[0033] Step S115: After completing the basic feature processing, different fields are combined in different ways to obtain composite features, which are then input into the structured data processing subnetwork to facilitate further feature extraction and fusion with other data sources;

[0034] The step S12 comprises:

[0035] Step S121: extracting original text data from the online complaint platform and emails, and cleaning the original text data;

[0036] Step S122: performing word segmentation processing on the cleaned text to split the continuous text into independent words or sub-word units, and then converting the word segmented text into a word embedding vector;

[0037] v word = Word2Vec(w), w is the input word, v word is the generated word embedding vector, which represents the numerical representation of word w in the embedding space; v is a high-dimensional vector; word is a subscript, indicating that this vector is associated with word w, and Word2Vec is the name of the preset model used to generate word embeddings;

[0038] Step S123: for the entire sentence, the word embedding vectors contained therein are merged into a sentence embedding vector by averaging or weighted averaging;

[0039] In the formula, v sentence is the generated sentence embedding vector, which represents the numerical representation of the entire sentence in the embedding space; sentence is a subscript, indicating that this vector is associated with the entire sentence; is a normalization factor used to calculate the average of all word embedding vectors in a sentence; is the embedding vector of the i-th word in the sentence; v is a vector representing the numerical representation of the i-th word; word i is a subscript indicating that the vector is associated with the i-th word in the sentence; N is the total number of words in the sentence;

[0040] Step S124: inputting the generated word embedding and / or sentence embedding vector into the text processing subnetwork for further semantic understanding and feature extraction;

[0041] The step S13 comprises:

[0042] Step S131: extracting original voice data from the telephone record, and removing background noise from the original voice data to obtain an audio signal;

[0043] Step S132: converting the audio signal into frequency domain features, and extracting higher-level speech features through Mel-frequency cepstral coefficients;

[0044] Where, X k is the frequency domain energy, which indicates the energy at the kth frequency; K is the number of Mel filter banks; n is the index of the cepstral coefficient, which is used to represent different MFCC features; N is the number of MFCC features finally extracted;

[0045] Step S133: Input the extracted frequency domain features or MFCC features into a preset speech recognition model to map the audio features into a phoneme sequence; wherein, for each time step, the speech recognition model calculates the probability p between the audio features and each phoneme:

[0046] p(y t |X t )=softmax(W*h t +b), where X t is the audio feature, h t is the hidden state, W is the weight matrix, b is the bias term, y t is the predicted probability distribution of the phoneme;

[0047] Step S134: decoding the phoneme sequence according to the context information to generate final text data to represent the actual content of the speech data;

[0048] Step S135: According to the steps S122-S123, the final text data is converted into word embedding vectors and / or sentence embedding vectors, and input into the speech processing subnetwork for further semantic understanding and feature extraction.

[0049] In a preferred embodiment, the step S14 includes:

[0050] Step S141: Integrate the features from structured data, text data and voice data by weighted averaging; the weighted averaging formula is as follows:

[0051] y fused =α*y stuctured +β*y text +γ*y speech , where y fused It is the final feature vector after fusion, which represents the unified representation obtained by integrating the data features from different sources. It contains the comprehensive information of structured data, text data and voice data, and is used for subsequent classification and regression tasks. structured is the feature vector output from the structured data processing subnetwork, consisting of numerical and categorical data; text is the feature vector output from the text processing subnetwork, representing the semantic information of the complaint content; speech is the feature vector output from the speech processing subnetwork, which contains the speech features extracted from the phone records; α, β, and γ are weighting coefficients used to control the contribution of each data source to the final fusion feature, satisfying the following conditions: α+β+γ=1;

[0052] Step S142: Map the fused high-dimensional features to a fixed output dimension and perform normalization processing, wherein the dimension of the fused feature vector is defined as d, and it is mapped to a fixed table format by the following formula:

[0053] Output Table =W final *y fused +b final , where Output Table It is the final output tabular data, which contains all processed and fused features. Each row corresponds to a complaint record, and each column corresponds to a specific feature. fused is the fused feature vector obtained in the previous step, which integrates the feature information of structured data, text data and speech data. It is obtained by weighted fusion of the features output by different sub-networks and contains the comprehensive feature representation of all data sources; Wfinal is the final weight matrix, which is used to transform the fused feature vector y fused Mapped to a fixed output dimension, responsible for converting high-dimensional fusion feature vectors into specific tabular data; b final is the final bias vector, which is related to W final Working together, they adjust the values ​​in the output table data so that the output does not depend entirely on the linear combination of the input features.

[0054] In a preferred embodiment, the step S20 includes:

[0055] Step S21: extracting two text field data of "complaint content summary" and "voice record summary" from the table data, and then performing word segmentation processing on the extracted text field data to split the continuous text into independent words or sub-word units;

[0056] Step S22: input the segmented text data into multiple pre-trained language models respectively, and each pre-trained language model generates a high-dimensional embedding representation of the corresponding text using its unique Transformer architecture;

[0057] Step S23: nonlinearly fuse the high-dimensional embedding representations generated by different pre-trained language models according to a preset multi-model embedding fusion formula; the multi-model embedding fusion formula is as follows:

[0058]

[0059] In the formula, h fused represents the fused text embedding representation vector, which combines the features from different pre-language training models; α1, α2, α3 represent weighting coefficients to control the impact of high-dimensional embedding representations of different pre-trained language models on the final result; represents the text embedding vector generated by the pre-language training model; hRoBERTa represents the text embedding vector generated by the pre-language training model; h ALBERT represents the text embedding vector generated by the pre-language training model; e hRoBERTa represents the exponential operation of the text embedding vector to increase the model's nonlinear processing ability for numerical values; log(h ALBERT +1) indicates the logarithmic processing of the text embedding vector, which is used to smooth the data and compress the dynamic range;

[0060] Step S24: fine-tune the fused embedding vector at the preset adaptive embedding layer node through a preset self-supervised learning formula to obtain a final embedding representation; wherein the self-supervised learning formula is as follows:

[0061] In the formula, h adaptiverepresents the text embedding vector after adaptive adjustment to adapt to specific tasks through nonlinear transformation; W represents the weight matrix of the adaptive embedding layer, which is used for linear transformation of the embedding vector; b represents the bias vector of the adaptive layer; tanh represents the hyperbolic tangent activation function, which introduces nonlinearity and limits the output between [-1,1]; represents the square root of the embedding vector dimension, used to normalize the input;

[0062] Step S25: extracting structured data from the table data, including numerical data and categorical data;

[0063] For numerical data, normalization is used to scale its values ​​to between 0 and 1; for categorical data, label encoding or one-hot encoding is used to convert categorical information into numerical representation.

[0064] In a preferred embodiment, the step S20 further includes:

[0065] Step S26: query relevant knowledge from the external domain knowledge base through the preset knowledge graph query node, and combine the generated knowledge vector with the text embedding vector through the formula of the preset graph convolutional network node to generate a knowledge-enhanced text embedding vector; wherein the formula is as follows:

[0066]

[0067] In the formula, h knowledge enhanced represents the text embedding vector after knowledge enhancement; W represents the weight matrix of the graph convolution network node; GCN() represents the graph convolution network operation, which is used to combine the adjacency matrix and the text embedding vector for convolution; A is the adjacency matrix of the knowledge graph, which represents the relationship between each node; deg(A) represents the degree matrix of the adjacency matrix A, which is used to normalize the adjacency matrix to ensure the stability of the convolution operation; b represents the bias vector to adjust the baseline value of the output;

[0068] Step S27: The text embedding vector and the structured data feature are fused in a preset feature interaction network node through a preset logarithmic function to generate a more complex high-dimensional feature representation; wherein the logarithmic function is as follows:

[0069] v interaction =log(h knowledge enhanced ⊙v structured +1),

[0070] In the formula, v interaction represents the feature vector after interaction, which combines the knowledge-enhanced text features and structured data features; h knowledge enhanced represents the knowledge-enhanced text embedding vector; v structuredRepresents a structured data feature vector; ⊙ represents an element-wise multiplication operation, which is used to realize the element-by-element interaction between text features and structured data features; log() represents a logarithmic operation, which is used to smooth the interaction results and handle the dynamic range;

[0071] Step S28: Process the fused feature vector through a preset self-attention mechanism node, and finally perform feature mapping at a preset fully connected layer node to generate a fixed-dimensional feature vector.

[0072] In a preferred embodiment, the encoder is as follows:

[0073] H = TransformerEncoder(v interaction , P), where H represents the hidden layer representation after being processed by the Transformer encoder, which contains rich semantic information extracted from the input features and is shared in the classification and regression tasks; the TransformerEncoder module processes the input feature representation through the self-attention mechanism and feedforward neural network to capture the relationship between features; v interaction represents the feature vector after interaction; P represents position encoding, which is used to maintain the position information of the input sequence to ensure that the model can correctly understand the features of each position in the sequence;

[0074] The classification task output header is as follows:

[0075] y class =softmax(W class ·H+b class ), where y class represents the output of the classification task, that is, the probability distribution of complaint categories. The probability of each category is calculated by the Softmax function to ensure that the sum of the probabilities of all categories is 1; W class is the weight matrix of the classification task output head, which is used to map the shared hidden layer representation H to the output space of the classification task; b class It is the bias item of the classification task output head, which is used to adjust the output result;

[0076] The regression task output header is as follows:

[0077] y reg =W reg ·H+b reg , where y reg is the output of the regression task, i.e., the complaint severity or urgency score; W reg is the weight matrix of the regression task output head, which is used to map the shared hidden layer representation H to the output value of the regression task; b reg It is the bias term of the regression task output head, which is used to linearly adjust the output result;

[0078] The joint loss function for optimizing the classification task and regression task in the multi-task learning process is as follows:

[0079]

[0080]

[0081] In the formula, It is the total loss function of multi-task learning to measure the overall performance of the model on classification and regression tasks; is the loss function of the classification task, which is used to measure the matching degree between the model's predicted probability of each category and the actual category; C is the number of categories, y i is an indicator variable of the actual category; is the loss function of the regression task, which is used to measure the difference between the model prediction value and the actual value. N is the number of samples. is the actual severity score; λ class and λ reg are the loss weights of the classification task and the regression task, respectively, which are used to balance the impact of the two tasks in the total loss;

[0082] The formula for dynamically adjusting the attention weight according to the context of the input text in the dynamic adaptive attention mechanism is as follows:

[0083]

[0084] In the formula, is the dynamic attention weight of the i-th position in the input sequence at time step t, which is calculated by the softmax function and changes with the content and position of the input text; Q t is the query matrix at time t, indicating the focus of the model on the text at the current time step; is the key matrix at the i-th position in the input sequence at time t, which is matched with the query matrix to find the most relevant information; is the value matrix of the i-th position in the input sequence at time t, containing the actual text information; n is the length of the input sequence, that is, the number of words in the text; d k is the dimension of the key matrix, which is used for normalization to prevent the gradient from disappearing due to excessive values.

[0085] In a preferred embodiment, the step S30 includes:

[0086] Step S31: In the data set, new synthetic samples are generated by SMOTE technology to balance the number of minority classes, so as to generate new synthetic samples between minority class samples to increase the number of minority classes; wherein the formula for generating new samples is:

[0087] x new =x i +δ·(x j -x i ), where x new is the generated synthetic sample, x i is a randomly selected minority class sample, x j From x i The sample is selected from the nearest neighbor samples of , δ is a random number between 0 and 1 to control the interpolation ratio between samples;

[0088] Step S32: introducing respective weights for each category in the cross-entropy-based loss function so that the model pays more attention to the minority category during training; wherein the weighted loss function is:

[0089] In the formula, is the weighted loss function, C is the total number of categories, and w i is the weight of category i, y i is the actual class label, is the probability of the category predicted by the model;

[0090] Step S33: During the model training process, the parameters of the model are trained by using the adaptive learning rate characteristics of the preset Adam optimization algorithm and introducing a weight decay mechanism; wherein the model parameters θ t The update formula in the tth iteration is:

[0091] In the formula, θ t is the model parameter after the tth iteration; η is the learning rate to control the step size of each parameter update; is the adaptive first-order moment estimate of the gradient, is the adaptive second-order moment estimate of the gradient; ∈ is a smoothing term to prevent division by zero errors; λ is a weight decay coefficient used to control the complexity of the model to reduce overfitting.

[0092] A second aspect of the present invention provides an automatic classification device for housing quality and safety complaints, comprising:

[0093] A data acquisition module is used to acquire housing quality and safety complaint data from the property management system, online complaint platform, emails and telephone records, and use a preset multimodal neural network to uniformly convert the housing quality and safety complaint data into tabular data; wherein the tabular data includes complaint ID, complaint type, complaint time and location, complaint content summary, processing time, complaint severity score and processing status;

[0094] A model building module, used to perform feature extraction and semantic enhancement on the table data, and to build a multi-task learning model combined with a dynamic adaptive attention mechanism; wherein the multi-task learning model includes a shared encoder and a classification task output head and a regression task output head; the classification task output head is used to map the hidden layer representation processed by the encoder to the probability distribution of various types of complaints, so as to be responsible for determining the specific category of the complaint; the regression task output head is used to map the hidden layer representation processed by the encoder to the output value of the regression task, so as to predict the severity or urgency score of the complaint;

[0095] A model training module is used to obtain the data set required for training the multi-task learning model, divide the data set into a training set, a validation set and a test set, and then train, validate and optimize the multi-task learning model;

[0096] A model evaluation module is used to evaluate and verify the trained multi-task learning model, and further tune the parameters of the model that does not meet the requirements based on the evaluation results, so that the performance of the model is balanced across all categories;

[0097] The model deployment module is used to deploy the trained multi-task learning model into the actual complaint processing system, receive complaint data, realize automatic classification and real-time severity scoring, and continuously optimize the system based on the feedback mechanism.

[0098] A third aspect of the present invention provides a terminal, which includes a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the various steps of the method for automatically classifying housing quality and safety complaints as described in any of the above-mentioned implementation methods.

[0099] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various steps of the method for automatically classifying housing quality and safety complaints as described in any of the above-mentioned implementation methods.

[0100] A fifth aspect of the present invention provides a computer program product, including a computer program or instructions, which, when processed and executed, implement the various steps of the method for automatically classifying housing quality and safety complaints as described in any of the above-mentioned implementation modes.

[0101] The automatic classification method and device for housing quality and safety complaints provided by the present invention have the following beneficial effects:

[0102] (1) By introducing a dynamic adaptive attention mechanism and a multi-task learning model, the present invention can adjust the attention weight in real time according to the context of the complaint text, significantly improving the classification accuracy of complex, multi-topic complaint texts;

[0103] (2) Through feature extraction and semantic enhancement technology, the present invention effectively captures key information in the complaint text, especially the professional terms and implicit semantics in housing quality complaints, making the severity score more accurate and ensuring that high-priority issues can be handled in a timely manner;

[0104] (3) A multimodal neural network is used to process data from multiple sources such as property management systems, online complaint platforms, emails, and phone records, achieving unified optimization and integration of structured and unstructured data, providing high-quality input features for the model;

[0105] (4) Through the feedback mechanism, new complaint data and processing results are continuously collected, and the model is regularly retrained and optimized so that the system can continuously adapt to new complaint types and changing processing requirements, maintaining long-term efficiency and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0107] Figure 1 A flow chart of the automatic classification method for housing quality and safety complaints provided by the present invention;

[0108] Figure 2 A multimodal neural network structure diagram in the automatic classification method for housing quality and safety complaints provided by the present invention;

[0109] Figure 3 A network structure diagram of feature extraction in the automatic classification method for housing quality and safety complaints provided by the present invention;

[0110] Figure 4 A network structure diagram of a semantically enhanced network in the automatic classification method for housing quality and safety complaints provided by the present invention;

[0111] Figure 5 A network structure diagram showing features in the automatic classification method for housing quality and safety complaints provided by the present invention;

[0112] Figure 6 A framework diagram of the automatic classification device for housing quality and safety complaints provided by the present invention. DETAILED DESCRIPTION

[0113] In order to make the purpose, technical solution and beneficial technical effect of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described in this specification are only for explaining the present invention, not for limiting the present invention.

[0114] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0115] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0116] Embodiment 1

[0117] In an embodiment of the present invention, a method for automatic classification of housing quality and safety complaints is provided, which is used to automatically process complaint data related to housing quality and safety, and introduces a dynamic adaptive attention mechanism and multi-task learning model technology to provide a more efficient, intelligent and safe complaint handling solution.

[0118] like Figure 1 As shown, the automatic classification method for housing quality and safety complaints includes steps S10-S50.

[0119] Step S10: Obtain housing quality and safety complaint data from the property management system, online complaint platform, email and telephone records, and use a preset multimodal neural network to uniformly convert the housing quality and safety complaint data into tabular data; wherein the tabular data includes complaint ID, complaint type, complaint time and location, complaint content summary, processing time, complaint severity score and processing status.

[0120] It should be noted that the "property management system" mentioned here can be an APP, mini-program, web page or other processing program, which can receive and record user complaints, including detailed records of housing quality complaints, such as complaint ID, complaint type, complaint time, location, problem description, processing status, etc., so its name can also be other. The term "property management system" does not limit the structured data storage carrier.

[0121] Combination Figure 2As shown, the multimodal neural network consists of multiple sub-networks, and each sub-network is responsible for processing data from different sources, including a structured data processing sub-network, a text processing sub-network, a speech processing sub-network and a feature fusion layer.

[0122] The structured data processing subnetwork is used to process the structured data from the property management system and is composed of a series of fully connected layers. It extracts and processes the features of the structured data according to the first preset formula. The first preset formula is as follows:

[0123] h1=ReLU(W1x+b1), h2=ReLU(W2h1+b2), y structured =W3h2+b3,

[0124] In the formula, x is the input structured data vector, W1, W2, W3 are weight matrices, b1, b2, b3 are biases, ReLU (Rectified Linear Unit) is the preset activation function used to introduce nonlinearity, and y structured is the output feature of the structured data processing sub-network.

[0125] The text processing subnetwork is used to process unstructured text data from online complaint platforms and emails. This subnetwork is usually built based on the Transformer architecture, such as the BERT (Bidirectional Encoder Representations from Transformers) model, and performs semantic understanding and feature extraction on the text according to the second preset formula. The second preset formula is as follows:

[0126] In the formula, K represents the key matrix (Key), which is extracted from the input word embedding or the output of the previous layer and is used to represent the features that need to be compared; Q represents the query matrix (Query), which is also extracted from the input word embedding or the output of the previous layer and is used to find features related to the key matrix K; the key matrix K measures the similarity by performing a dot product with the query matrix Q; V represents the value matrix (Value), which corresponds to the actual content of the input and generates the final output through weighted combination; Attention represents the output of the attention mechanism, which is used to perform weighted averaging on the value matrix V; softmax represents a normalization function, which is used to convert the dot product result of the query matrix Q and the key matrix K into a probability distribution, so that the sum of the attention weights is 1; is the scaled dot product of the query matrix Q and the key matrix K, d k is the dimension of the key vector, the scaling factor Used to avoid gradient vanishing due to large values; T is the matrix transpose.

[0127] The speech processing subnetwork is used to process the speech data extracted from the phone records. This subnetwork usually converts the speech data into text first, and then processes the text, similar to the text processing subnetwork, that is, the text is processed accordingly through the second preset formula. The feature extraction formula of this subnetwork is consistent with that of the text processing subnetwork, and it also uses a multi-head attention mechanism and a fully connected layer.

[0128] Specifically, in combination with the above description of each sub-network in the multimodal neural network, step S10 includes steps S11-S14.

[0129] Step S11: Perform structured data processing on the complaint data of the property management system, and input the obtained structured data into the structured data processing sub-network. Step S11 includes steps S111-S115.

[0130] Step S111: extract structured data containing detailed records of housing quality complaints from the property management system, including complaint ID, complaint type, complaint time and location, problem description, and processing status.

[0131] Step S112: Identify and process missing values ​​and outliers in the extracted data; including: (1) Missing value processing: Identify and process missing values ​​in the data. For example, this can be done by mean filling, forward filling, or removing records with a large number of missing values; (2) Outlier detection: Identify and process outliers, which may be caused by input errors or equipment failures.

[0132] Step S113: Encode the categorical data to obtain categorical features. For categorical data from the property management system, such as "complaint type" (e.g., water leakage, cracks, power supply problems, etc.), one-hot encoding can be used. For example, assuming that there are four possibilities for complaint types:

[0133] (1) Water leakage: 1,0,0,0

[0134] (2) Crack: 0,1,0,0

[0135] (3) Power supply problem: 0,0,1,0

[0136] (4) Others: 0, 0, 0, 1.

[0137] For categories with order (such as complaint priority: low, medium, high), label encoding or target encoding methods are used to preserve their order information.

[0138] Step S114: Split the timestamp data (such as complaint submission time, processing start time, etc.) into multiple time fields including "year, month, day, hour, minute", and perform periodic feature processing to capture the periodic characteristics of time. Process the time field through periodic feature processing (such as sine and cosine conversion). Take the month as an example:

[0139] In the formula, Month sin Moth sin Represents the sinusoidal characteristics of the month, Month cos Represents the cosine feature of the month; sin represents the sine function, inputs an angle (radian) and returns the sine value of the corresponding angle; cos represents the cosine function, inputs an angle (radian) and returns the cosine value of the corresponding angle; Month is the input month, usually ranging from 1 to 12, indicating the number of months in a year; the denominator 12 means that the month is divided into 12 equal parts to ensure that the angle mapped to the month is between 0 and 2π (i.e. a complete circle).

[0140] Month sin Month sin and Month cos Month cos These two formulas are used to transform monthly data into periodic features. This transformation allows the model to better understand the cyclical nature of time, such as certain events that may recur in specific months or seasons. and Mapping the linear number of months to a circular cycle allows the model to perceive the periodicity of time. For example, January and December are adjacent in this representation, reflecting the natural periodicity of time. This processing method can capture the periodic characteristics of time, such as certain problems may occur more frequently in specific months.

[0141] Step S115: After completing the basic feature processing, perform feature combinations in different ways on different fields to obtain composite features, which are then input into the structured data processing subnetwork for further feature extraction and fusion with other data sources. For example, a new interactive feature is generated by combining "complaint type" and "processing time" to represent the average processing time required for a specific type of complaint. New features are generated by combination, such as composite features of "complaint type" and "complaint priority", which can help the network capture more complex patterns. Finally, these processed and optimized structured data are input into the structured data processing subnetwork for further feature extraction and fusion with other data sources (such as text and voice).

[0142] Step S12: Perform unstructured data processing on the complaint data of both the online complaint platform and emails, and input the generated word embedding vectors and / or sentence embedding vectors into the text processing sub-network. Step S12 includes steps S121 - S124.

[0143] Step S121: Extract the original text data from the online complaint platform and emails, and clean the original text data.

[0144] Specifically, first extract the original text data from the online complaint platform and emails. These texts usually contain detailed complaint information from users, including problem descriptions, time and location, user feedback, etc. Then, clean the text data. The specific steps include:

[0145] (1) Remove noise: Remove irrelevant information such as HTML tags, special characters (such as @, #, $, etc.), and punctuation marks.

[0146] (2) Case normalization: Convert all texts to lowercase (or keep the original format as required).

[0147] (3) Remove stop words: Remove common stop words (such as "of", "is", "in", etc.), which usually contribute less to semantic understanding.

[0148] Step S122: Perform word segmentation on the cleaned text to split the continuous text into independent words or sub-word units, and then convert the segmented text into word embedding vectors.

[0149] Specifically, word segmentation is to split the continuous text into independent words or sub-word units. This step usually uses Chinese word segmentation tools (such as Jieba segmentation), and convert the segmented text into word embedding vectors. Word embedding is a technique that maps words into a continuous vector space, making similar words close to each other in the vector space. Here, an existing Word2Vec pre-trained model can be used to convert each word into a high-dimensional vector: v word = Word2Vec(w), where w is the input word, such as the words "complaint" or "crack" mentioned in the text; v word is the generated word embedding vector, representing the numerical representation of the word w in the embedding space; v is a high-dimensional vector, usually high-dimensional, such as 100-dimensional, 300-dimensional, etc.; word is a subscript indicating that this vector is associated with the word w, and Word2Vec is the name of the pre-set model for generating word embeddings. Among them, Word2Vec is a commonly used word embedding technology that can map words into a continuous vector space. w is the input word, such as the words "complaint" or "crack" mentioned in the text.

[0150] Step S123: for the entire sentence, the word embedding vectors contained therein are merged into a sentence embedding vector by averaging or weighted averaging;

[0151] In the formula, v sentence is the generated sentence embedding vector (sentence embedding representation), which represents the numerical representation of the entire sentence in the embedding space; sentence is a subscript, indicating that this vector is associated with the entire sentence; is a normalization factor used to calculate the average of all word embedding vectors in a sentence; is the embedding vector of the i-th word in the sentence; v is a vector representing the numerical representation of the i-th word; word i is a subscript indicating that the vector is associated with the i-th word in the sentence; N is the total number of words in the sentence, which is used as the average value when calculating the sentence embedding.

[0152] Step S124: Input the generated word embedding and / or sentence embedding vector into the text processing subnetwork for further semantic understanding and feature extraction.

[0153] Step S13: Perform voice data processing on the complaint data recorded by the telephone, generate text output, convert the generated text data into a word embedding vector, and then input the generated word embedding vector into the voice processing sub-network. Step S13 includes steps S131-S135.

[0154] Step S131: extracting original voice data from the telephone record, and removing background noise from the original voice data to obtain an audio signal.

[0155] The voice data in the phone records are usually unstructured audio files. In order to enable these data to be processed by neural networks, first, the original voice data is extracted from the phone records. These voice files are usually stored in WAV, MP3 and other formats, and contain user complaints, conversation records, etc.

[0156] Next, the speech data is preprocessed, specifically by removing the background noise in the speech signal through a filter. The formula involved in the filter is as follows: y(t) = x(t)*h(t); where x(t) represents the original speech signal, h(t) is the impulse response of the filter, and y(t) is the filtered speech signal, which retains the useful part of the speech and removes the noise. The function of the filter is to retain the useful signal within the speech frequency range and remove the noise in other frequency ranges.

[0157] Step S132: Convert the audio signal into frequency domain features, and extract higher-level speech features through Mel-frequency cepstral coefficients.

[0158] Specifically, first, the time domain signal can be converted into a frequency domain representation by short-time Fourier transform (STFT). The short-time Fourier transform formula is as follows:

[0159] Where X(ω,t) is the frequency domain representation, which indicates the frequency components of the signal at different time points; x(τ)x(τ) is the original time domain signal, i.e., the speech data; w(t-τ)w(t-τ) is the preset window function, which is used to intercept the signal in a short time for Fourier transform; w is the frequency, which indicates the component of the signal at a certain frequency; t is the time, which indicates the frequency distribution of the signal at a specific time point.

[0160] The processed audio signal is then used to extract higher-level speech features through Mel-frequency cepstral coefficients (MFCC), which can better represent the phonetic information in the speech.

[0161] Where, X k is the frequency domain energy, which indicates the energy at the kth frequency; K is the number of Mel filter banks; n is the index of the cepstral coefficient, which is used to represent different MFCC features; N is the number of MFCC features finally extracted;

[0162] Step S133: Input the extracted frequency domain features or MFCC features into a preset speech recognition model to map the audio features into a phoneme sequence. It should be noted that the implementation principle and specific implementation of the speech recognition model can refer to the prior art and will not be repeated here. For each time step, the speech recognition model calculates the probability p between the audio features and each phoneme:

[0163] p(y t |X t )=softmax(W*h t +b), where X t is the audio feature, h t is the hidden state, W is the weight matrix, b is the bias term, y t is the predicted probability distribution of phonemes; the Softmax function is an activation function commonly used in multi-classification problems, which can convert a real number vector into a probability distribution.

[0164] Step S134: Decode the phoneme sequence according to the context information to generate final text data to represent the actual content of the speech data. This text data is a sequence that represents the actual content of the speech data. For example, the speech recognition model may recognize a speech as: "The roof is leaking and needs urgent repair."

[0165] Step S135: According to steps S122-S123, the final text data is converted into word embedding vectors and / or sentence embedding vectors, and input into the speech processing subnetwork for further semantic understanding and feature extraction. That is, the generated text data is converted into word embedding vectors. This part of the steps is the same as the text processing in steps S122-S123. First, word segmentation is performed, and then the word embedding model (such as Word2Vec, BERT, etc.) is used to convert the words into numerical vectors.

[0166] Step S14: Integrate the features from the structured data, text data and voice data through the feature fusion layer of the multimodal neural network to form a unified feature representation based on a table. Step S14 includes steps S141-142.

[0167] Step S141: Integrate the features from structured data, text data and voice data by weighted averaging; the weighted averaging formula is as follows:

[0168] y fused =α*ys tructured +β*y text +γ*y speech , where y fused It is the final feature vector after fusion, which represents the unified representation obtained by integrating the data features from different sources. It contains the comprehensive information of structured data, text data and voice data, and is used for subsequent classification, regression tasks or other machine learning tasks; structured is the feature vector output from the structured data processing subnetwork, which consists of numerical and categorical data and contains data features from the property management system, such as complaint ID, complaint type, processing time, etc.; text is the feature vector output from the text processing subnetwork, which contains text features extracted from the online complaint platform and emails. After text processing (such as word segmentation, word embedding, and semantic extraction), these feature vectors represent the semantic information of the complaint content; speech It is the feature vector output from the speech processing subnetwork, which contains the speech features extracted from the phone records. The speech data is converted into text features after speech recognition and processing, and then the semantic information is further extracted to generate this feature vector; α, β, γ are weighting coefficients used to control the contribution of each data source to the final fusion feature. Their values ​​are usually hyperparameters, which can be optimized through cross-validation and other methods during training and meet the following conditions: α+β+γ=1.

[0169] Step S142: Map the fused high-dimensional features to a fixed output dimension and perform normalization to ensure that the output features can be directly used for table representation. The dimension of the fused feature vector is defined as d, and it is mapped to a fixed table format using the following formula:

[0170] Output Table =W final *y fused +b final , where Output Table It is the final output tabular data, which contains all processed and fused features. Each row corresponds to a complaint record, and each column corresponds to a specific feature, such as complaint ID, complaint type, processing time, severity score, etc. These data are finally presented in the form of a table to facilitate subsequent analysis and processing; fused is the fused feature vector obtained in the previous step, which integrates the feature information of structured data, text data and speech data. It is obtained by weighted fusion of the features output by different sub-networks and contains the comprehensive feature representation of all data sources; W final is the final weight matrix, which is used to transform the fused feature vector y fused Mapped to a fixed output dimension, the dimension of this matrix is ​​usually [output feature number, fusion feature dimension], which is responsible for converting the high-dimensional fusion feature vector into specific tabular data; b final is the final bias vector, which is related to W final Working together, they adjust the values ​​of each item in the output table data. By adding the bias term, the flexibility of the model is increased so that the output does not completely depend on the linear combination of the input features.

[0171] For example, the final table looks like this:

[0172]

[0173]

[0174] In this table, each column shows the integrated multimodal data. "Complaint ID" is a unique identifier provided by the property management system; "Complaint Type" is a classification of complaint events, such as "Leakage", "Crack" or "Power Supply Problem"; "Complaint Time" and "Location" columns show the time and location of the complaint, which are derived from the timestamp and location fields in the property management system. "Complaint Content Summary" is a summary of the detailed complaint information in the online complaint platform and email; "Processing Time" column shows the time taken from complaint submission to completion. "Complaint Severity Score" is the result predicted by the multimodal neural network, combining the features extracted by each sub-network; "Processing Status" column shows the progress of the current complaint. "Voice Record Summary" is a text summary of the processed voice data in the phone record. Finally, "Merged Features 1" and "Merged Features 2" are comprehensive features generated by the feature fusion layer, which further enhance the model's ability to understand and analyze the data. These features integrate information from different data sources through neural networks to help better complaint handling and decision support.

[0175] Therefore, the present invention first proposes a multimodal neural network technology that integrates data from multiple sources, and solves the problem of poor field adaptability in view of the particularity of the field of housing quality and safety complaints. By integrating multiple data sources such as property management systems, online complaint platforms, emails, and telephone records, this method introduces multimodal neural networks in this field for the first time to perform data preprocessing and feature extraction to generate high-quality model input features. The innovation of this technology is that it introduces multimodal data integration in the processing of housing quality and safety complaints for the first time, which not only enhances the model's ability to understand complex scenarios, but also improves the comprehensiveness and accuracy of data processing, avoiding the problems of incomplete information and insufficient adaptability caused by a single data source in the existing system. The present invention uses a multimodal neural network to process data from multiple sources such as property management systems, online complaint platforms, emails, and telephone records, realizes the unified optimization and integration of structured data and unstructured data, and provides high-quality input features for the model.

[0176] Step S20: Feature extraction and semantic enhancement are performed on the table data to construct a multi-task learning model combined with a dynamic adaptive attention mechanism. The multi-task learning model includes a shared encoder and a classification task output head and a regression task output head. The classification task output head is used to map the hidden layer representation processed by the encoder to the probability distribution of various types of complaints to determine the specific category of the complaint. The regression task output head is used to map the hidden layer representation processed by the encoder to the output value of the regression task to predict the severity or urgency score of the complaint.

[0177] It can be understood that a table integrating housing quality complaint data from multiple sources has been obtained through step S10. This table includes structured data (such as complaint ID, complaint type, complaint time, etc.), text data (such as complaint content summary), and text data converted from voice recordings (such as voice recording summary). Next, feature extraction will be performed based on this table data, and semantic enhancement will be used to improve the model's ability to understand the complaint text.

[0178] Combination Figure 3 , Figure 4 and Figure 5 As shown, specifically, step S20 includes steps S21-S28.

[0179] Step S21: extract the two text field data of "complaint content summary" and "voice record summary" from the table data, and then perform word segmentation on the extracted text field data to split the continuous text into independent words or subword units. Here, "complaint content summary" and "voice record summary" refer to the data names from the original text and voice, which can be referred to by other nouns, so these two nouns are not used as limitations on the two text field data.

[0180] Step S22: The segmented text data is input into a plurality of pre-trained language models respectively, and each pre-trained language model generates a high-dimensional embedding representation of the corresponding text using its unique Transformer architecture.

[0181] Step S23: nonlinearly fuse the high-dimensional embedding representations generated by different pre-trained language models according to the preset multi-model embedding fusion formula to fully utilize the advantages of these pre-trained language models. The pre-trained language model can refer to the existing technology, such as BERT, GPT, XLNet, ALBERT, ROBERTa, etc., and its implementation principle and specific implementation method will not be repeated here.

[0182] Among them, the multi-model embedding fusion formula is as follows:

[0183]

[0184] In the formula, h fused represents the fused text embedding representation vector, which combines the features from different pre-language training models; α1, α2, α3 represent weighting coefficients to control the impact of high-dimensional embedding representations of different pre-trained language models on the final result; represents the text embedding vector generated by the pre-language training model BERT; hRoBERTa (i.e. h ROBERTa ) represents the text embedding vector generated by the pre-language training model ROBERTa; h ALBERTrepresents the text embedding vector generated by the pre-language training model ALBERT; e hRoBERTa represents the exponential operation of the text embedding vector to increase the model's nonlinear processing ability for numerical values; log(h ALBERT +1) represents the logarithmic processing of the text embedding vector, which is used to smooth the data and compress the dynamic range. Therefore, this formula performs a weighted fusion of the embeddings of different models through square, cubic and logarithmic operations to generate a richer text feature representation.

[0185] Step S24: The fused embedding vector is embedded in the preset adaptive embedding layer node (see Figure 3 ) is fine-tuned to obtain the final embedding representation, which makes it better suited to subsequent classification and regression tasks; the self-supervised learning formula is as follows:

[0186] In the formula, h adaptive represents the text embedding vector after adaptive adjustment to adapt to specific tasks through nonlinear transformation; W represents the weight matrix of the adaptive embedding layer, which is used for linear transformation of the embedding vector; b represents the bias vector of the adaptive layer; tanh represents the hyperbolic tangent activation function, which introduces nonlinearity and limits the output between [-1,1]; Represents the square root of the embedding vector dimension, which is used to normalize the input to avoid the impact of too large or too small values. This formula further enhances its expressiveness and task adaptability by taking the derivative and performing sine function processing on the fused embedding vector, and finally outputs the "final embedding representation".

[0187] Step S25: extracting structured data from the table data, including numerical data and categorical data;

[0188] For numerical data, normalization is used to scale its values ​​to between 0 and 1; for categorical data, label encoding or one-hot encoding is used to convert categorical information into numerical representation.

[0189] Combination Figure 5 As shown in the figure, structured data features such as "complaint type", "complaint time", "processing time", "complaint severity score" and other data are first extracted. These data exist in numerical or categorical form. Specifically, for numerical data (such as processing time), its value is scaled to between 0 and 1 through normalization so that the model can better process and learn. For categorical data (such as complaint type), the category information is converted into a numerical representation through label encoding or one-hot encoding. The processed structured data features will be fused with text features in subsequent steps.

[0190] Step S26: Combination Figure 4As shown in the figure, the relevant knowledge is queried from the external domain knowledge base (such as professional vocabulary and rules in the field of architecture) through the preset knowledge graph query node (), and the generated knowledge vector is combined with the text embedding vector through the formula of the preset graph convolutional network node to generate a knowledge-enhanced text embedding vector; wherein, the formula is as follows:

[0191]

[0192] In the formula, h knowledge enhanced represents the text embedding vector after knowledge enhancement; W represents the weight matrix of the graph convolutional network node; GCN() represents the graph convolutional network operation, which is used to combine the adjacency matrix and the text embedding vector for convolution; A is the adjacency matrix of the knowledge graph, which represents each Figure 4 deg(A) represents the degree matrix of the adjacency matrix A, which is used to normalize the adjacency matrix to ensure the stability of the convolution operation; b represents the bias vector to adjust the output baseline value. Therefore, through the graph convolution network, domain knowledge and text features are convolutionally fused, and the generated knowledge-enhanced embedding is more suitable for processing complaint texts involving specific domain terms.

[0193] The enhanced text embedding is achieved through Figure 4 The “hierarchical attention mechanism” node in the code is further processed to capture the implicit semantics in the text, strengthen the understanding of the key information in the complaint text, and finally output the “enhanced semantic representation”.

[0194] Step S27: The text embedding vector and the structured data features are fused in the preset feature interaction network node through a preset logarithmic function to generate a more complex high-dimensional feature representation. That is, in the feature representation stage, the text features are fused with the structured data features to generate the final model input. Figure 5 As shown in Figure 1, the text embedding representation (i.e., text embedding vector) and structured data features are first fused in the feature interaction network node. The key innovation of this process is to use logarithmic functions combined with feature interaction methods to generate more complex high-dimensional feature representations.

[0195] The logarithmic function is as follows:

[0196] v interaction =log(h knowledge enhanced ⊙v structured +1),

[0197] In the formula, v interaction represents the feature vector after interaction, which combines the knowledge-enhanced text features and structured data features; h knowledge enhanced represents the knowledge-enhanced text embedding vector; v structuredrepresents the structured data feature vector; ⊙ represents the element-by-element multiplication operation, which is used to realize the element-by-element interaction between text features and structured data features; log() represents the logarithmic operation, which is used to smooth the interaction results and handle the dynamic range. This formula enables the model to capture the deep relationship between text features and structured data features through logarithmic operations and feature interactions, and generate more expressive feature representations.

[0198] Step S28: Process the fused feature vector through the preset self-attention mechanism node, and finally perform feature mapping at the preset fully connected layer node to generate a fixed-dimensional feature vector. Figure 5 The self-attention mechanism nodes in the model are processed to identify and strengthen important feature information, ensuring that the model can effectively focus on key information. Finally, feature mapping is performed in the fully connected layer nodes to generate a "fixed-dimensional feature vector" for subsequent multi-task learning model input.

[0199] In this embodiment, the multi-task learning model consists of a shared Transformer encoder and two task-specific output heads, one for classification and one for regression. The key to this architecture is to extract common features through the shared encoder while allowing each task to be optimized on its own output head.

[0200] The Transformer encoder is as follows:

[0201] H = TransformerEncoder(v interaction , P), where H represents the hidden layer representation after being processed by the Transformer encoder, which contains rich semantic information extracted from the input features and is shared in the classification and regression tasks; the TransformerEncoder module processes the input feature representation through the self-attention mechanism and feedforward neural network, so that the model can focus on important information to capture the relationship between features; v interaction Represents the feature vector after interaction, which contains comprehensive information of text and structured data; P in this formula represents position encoding, which is used to maintain the position information of the input sequence to ensure that the model can correctly understand the features of each position in the sequence.

[0202] The classification task is responsible for determining the specific category of the complaint, such as "water leakage", "cracks", "power supply problems", etc. To achieve classification, a classification output head is added after the shared encoder of the model, which maps the hidden layer representation to the probability distribution of each type of complaint.

[0203] Among them, the classification task output head is as follows:

[0204] y class=softmax(W class ·H+b class ), where y class represents the output of the classification task, that is, the probability distribution of complaint categories. The probability of each category is calculated by the Softmax function to ensure that the sum of the probabilities of all categories is 1; W class is the weight matrix of the classification task output head, which is used to map the shared hidden layer representation H to the output space of the classification task; b class It is the bias item of the classification task output head, which is used to adjust the output results.

[0205] The regression task is responsible for predicting the severity or urgency score of the complaint (e.g., a score from 1 to 10). To implement the regression task, a regression output head is added after the shared Transformer encoder.

[0206] Among them, the regression task output head is as follows:

[0207] y reg =W reg ·H+b reg , where y reg is the output of the regression task, i.e., the complaint severity or urgency score; W reg is the weight matrix of the regression task output head, which is used to map the shared hidden layer representation H to the output value of the regression task; b reg It is the bias term of the regression task output head, which is used to linearly adjust the output results.

[0208] In order to optimize the classification and regression tasks simultaneously, the model adopts a joint loss function. The joint loss function consists of classification loss and regression loss, and each loss function adjusts its weight according to actual needs, thereby achieving a good balance in the multi-task learning process.

[0209] Among them, the joint loss function for optimizing the classification task and regression task in the multi-task learning process is as follows:

[0210]

[0211]

[0212] In the formula, It is the total loss function of multi-task learning to measure the overall performance of the model on classification and regression tasks; is the loss function for classification tasks. The cross entropy loss function is usually selected to measure the matching degree between the model's predicted probability for each category and the actual category. C is the number of categories, y i is an indicator variable of the actual category (one-hot encoding); is the loss function for regression tasks. The mean square error (MSE) loss function is usually selected to measure the difference between the model prediction value and the actual value. N is the number of samples. is the actual severity score. class λ class and λ reg are the loss weights for the classification task and the regression task, respectively, which are used to balance the impact of the two tasks in the total loss. These weights can be adjusted experimentally to optimize the overall performance of the model.

[0213] In this embodiment, in the process of building a multi-task learning model, in order to further improve the performance of the model in processing complex, multi-topic complaint texts, a dynamic adaptive attention mechanism is introduced in the multi-task learning model. This mechanism enables the model to adapt to the feature changes of different text contents in real time by dynamically adjusting the attention weights, so as to more accurately identify and process the key information in the complaint text. Traditional attention mechanisms usually use fixed weights when processing all input sequences, which is not flexible enough for complex and multi-topic texts. To solve this problem, the dynamic adaptive attention mechanism allows the model to dynamically adjust the attention weights according to the context of the input text, so as to process different text paragraphs or topics more flexibly.

[0214] Among them, the formula for dynamically adjusting the attention weight according to the context of the input text in the dynamic adaptive attention mechanism is as follows:

[0215]

[0216] In the formula, is the dynamic attention weight of the i-th position in the input sequence at time step t, which is calculated by the softmax function and changes with the content and position of the input text; Q t is the query matrix (Query) at time t, which indicates the focus of the model on the text at the current time step; is the key matrix (Key) of the i-th position in the input sequence at time t, which is matched with the query matrix to find the most relevant information; is the value matrix (Value) of the i-th position in the input sequence at time t, containing the actual text information; n is the length of the input sequence, that is, the number of words in the text; d k is the dimension of the key matrix, which is used for normalization to prevent the gradient from disappearing due to excessive values.

[0217] The dynamic adaptive attention mechanism introduces the time step t so that the attention weight of each time step It can be adjusted in real time according to the context of the current text. This means that when processing text input at different times, the model can dynamically adjust the attention paid to text in different locations. For example, when the model processes key terms such as "load-bearing wall" or "foundation settlement", the mechanism will automatically increase the attention paid to these terms, making the model more flexible and intelligent.

[0218] Compared with the traditional static attention mechanism, this formula introduces dynamic weights Real-time adjustment of attention is achieved, which truly embodies the "dynamic" feature. This improvement enables the model to capture key information more accurately when dealing with multi-topic and complex texts, and improves the model's classification and severity prediction capabilities. By dynamically adjusting attention at each time step, the model can flexibly respond to multiple topics in complex complaint texts. This mechanism enables the model to adaptively adjust its focus when dealing with different paragraphs or topics, thereby improving the performance of the model in complex scenarios. For example, when multiple issues are involved at the same time, the model can automatically identify and distinguish the importance of different topics to ensure that the model's predictions are more accurate.

[0219] Therefore, on the basis of data preprocessing, the present invention further proposes a multi-task learning model combined with a dynamic adaptive attention mechanism, and introduces this mechanism for the first time in the classification of housing quality complaints to cope with complaints with diverse professional terms and complex semantics. This innovative model improves the recognition accuracy of the model through feature extraction and semantic enhancement, especially the introduction of a dynamic adaptive attention mechanism. Compared with traditional NLP models, the dynamic adaptive attention mechanism allows the model to dynamically adjust the attention weight according to the context of the input text, so as to more accurately identify and process different text paragraphs or topics, significantly improve the accuracy of identifying professional terms and processing complex scenarios, and effectively solve the shortcomings of existing technologies in terms of recognition accuracy.

[0220] By introducing a dynamic adaptive attention mechanism and a multi-task learning model, the present invention can adjust the attention weight in real time according to the context of the complaint text, significantly improving the classification accuracy of complex, multi-topic complaint texts. In addition, through feature extraction and semantic enhancement technology, the present invention effectively captures the key information in the complaint text, especially the professional terms and implicit semantics in housing quality complaints, making the severity score more accurate and ensuring that high-priority issues can be handled in a timely manner.

[0221] Step S30: Obtain the data set required for training the multi-task learning model, divide the data set into a training set, a validation set and a test set, and then train, validate and optimize the multi-task learning model.

[0222] The data set required for the improved NLP model constructed by the present invention is divided into a training set, a validation set and a test set. These data sets will ensure that the model can be fully trained, validated and evaluated on different data sets.

[0223] Training set: used for model parameter learning. Usually contains 70-80% of the dataset to ensure that the model can capture the basic patterns of the data.

[0224] Validation set: used for adjusting and selecting model hyperparameters. Usually accounting for 10-15% of the dataset, the validation set can prevent model overfitting and select the optimal model parameters.

[0225] Test set: used to evaluate the performance of the model on unseen data. Usually 10-15% of the dataset to ensure the generalization ability of the model.

[0226] Specifically, step S30 includes steps S31-S33.

[0227] Step S31: In the data set, new synthetic samples are generated by SMOTE technology to balance the number of minority classes, so as to generate new synthetic samples between minority class samples to increase the number of minority classes.

[0228] When dividing data, it is particularly important to pay attention to the imbalance of data categories. For example, if the number of samples of certain complaint types (such as "power supply problems") is much smaller than other categories, the model may perform poorly in predicting these categories. To solve this problem, the SMOTE (Synthetic Minority Over-sampling Technique) technology can be used in the data preparation stage. Specifically, the SMOTE technology generates new synthetic samples between minority class samples to increase the number of minority classes.

[0229] Among them, the formula for generating new samples is:

[0230] x new =x i +δ·(x j -x i ), where x new is the generated synthetic sample, x i is a randomly selected minority class sample, x j From x i The samples are selected from the nearest neighbor samples of , and δ is a random number between 0 and 1 to control the interpolation ratio between samples. In this way, SMOTE effectively increases the number of samples in the minority class, thereby balancing the data distribution.

[0231] Step S32: introducing respective weights for each category in the cross entropy-based loss function so that the model pays more attention to the minority category during the training process; wherein, the specific implementation method is to introduce weights for each category in the cross entropy loss function, and the weighted loss function is:

[0232] In the formula, is the weighted loss function, C is the total number of categories, and w i is the weight of category i, y i is the actual category label (one-hot encoding), is the probability of the category predicted by the model. By adjusting the weights of each category, the weighted loss function can force the model to impose a greater penalty on errors in the minority category during training, thereby effectively improving the model's prediction performance on the minority category. These methods help balance the data distribution, thereby improving the model's prediction performance on different categories and ensuring that the model does not perform poorly when processing minority category samples due to a small number of samples.

[0233] Step S33: During the model training process, the parameters of the model are trained by using the adaptive learning rate characteristics of the preset Adam optimization algorithm and introducing a weight decay mechanism. After the data preparation is completed, the model training and optimization phase is entered. The goal of this phase is to maximize the performance of the model through reasonable training strategies and optimization algorithms, so that it can perform well in both classification and regression tasks. During the model training process, the AdamW (Adaptive MomentEstimation) optimization algorithm was first selected. AdamW combines adaptive learning rate and weight decay technology. Its core principle is to combine the adaptive learning rate characteristics of Adam optimization, and to control the complexity of the model and reduce overfitting by introducing a weight decay mechanism.

[0234] Among them, the parameters of the model θ t The update formula in the tth iteration is:

[0235] In the formula, θ t is the model parameter after the tth iteration; η is the learning rate to control the step size of each parameter update; is the adaptive first-order moment estimate of the gradient, is the adaptive second-order moment estimate of the gradient; ∈ is a smoothing term to prevent division by zero errors; λ is a weight decay coefficient used to control the complexity of the model to reduce overfitting. Through this formula, the model can adaptively adjust the learning rate while controlling the growth of parameters, making the training more stable.

[0236] It should be noted that, in the actual training process, reasonable hyperparameters such as learning rate and batch size are first set. The learning rate directly affects the convergence speed and stability of the model, while the batch size affects the stability of the gradient estimation and the training efficiency. In this embodiment, the present invention sets the learning rate to 0.001 and the batch size to 32. The settings of these hyperparameters are tuned by the validation set to ensure that the model can converge quickly and stably during the training process.

[0237] During the entire training process, the model calculates the output through forward propagation and calculates the loss of classification and regression tasks using the defined joint loss function. Then, the parameters are updated through the backpropagation algorithm to minimize the loss. After each iteration, the performance of the model on the validation set is evaluated to ensure that it is not overfitting or underfitting.

[0238] The iteration results are shown in the following table:

[0239]

[0240]

[0241] As can be seen from the table above, as the number of iterations increases, the loss values ​​on the model's training set and validation set gradually decrease, the accuracy of the classification task gradually increases, and the mean square error (MSE) of the regression task gradually decreases. This shows that the model is continuously optimized during the training process, gradually showing better performance on the validation set, and finally achieving a relatively ideal effect at the 10th iteration. The performance of the model on the training set and validation set tends to be stable, indicating that the model has effectively learned the pattern of the data and has good generalization ability. Through reasonable training and optimization, the model has achieved the expected results in both classification and regression tasks.

[0242] Therefore, in order to further improve the practical application effect of the model, the present invention proposes for the first time a technology combining multi-task learning with model tuning in the process of model training and optimization, and solves the problem of cumbersome processing flow by extracting common features through a shared encoder. The innovation of this technology is that, through the construction of a multi-task learning model, the model can simultaneously perform classification and regression tasks, realize the category judgment and severity scoring of complaints, and combine the dynamic adaptive attention mechanism of the model to continuously optimize and improve performance during the training process. This process uses reasonable training strategies and optimization algorithms to enable the model to converge quickly and stably, avoiding the cumbersome process of manual intervention and confirmation in the existing system and improving processing efficiency.

[0243] Step S40: Perform model evaluation and verification on the trained multi-task learning model, and further tune the parameters of the model that does not meet the requirements based on the evaluation results to make the model perform evenly across all categories.

[0244] During the model evaluation phase, a series of evaluation indicators were used, including weighted precision, generalized F1-score, and category-sensitive recall, which aim to more accurately reflect the performance of the model in handling housing quality complaint classification tasks, especially when dealing with category-imbalanced data. The following is a detailed explanation of each formula.

[0245] The weighted precision formula is as follows:

[0246]

[0247] In this formula, C is the total number of categories (such as "leakage", "crack", "power supply problem", etc.); w i is the weight of category i, which is used to reflect the importance of each category in the overall evaluation. These weights can be set according to the sample ratio of the category or business needs; TP i is the true class number of category i, that is, the number of samples correctly predicted by the model as positive; TN i is the number of true negative classes of category i, that is, the number of samples correctly predicted by the model as negative classes; FP i is the number of false positives for category i, that is, the number of samples that the model incorrectly predicts as positive; FN i is the number of false negatives for category i, that is, the number of samples that the model incorrectly predicts as negative. This formula introduces a weight coefficient w based on the traditional precision rate. i , which makes it possible to consider the importance of different categories when evaluating the accuracy, especially giving higher weights to minority categories, thereby avoiding the situation where the model is biased towards the majority class.

[0248] The generalized F1-score formula is as follows:

[0249]

[0250] In this formula, C is the total number of categories; w i The weight of category i is used to adjust the sensitivity of F1-score to different categories;

[0251] Precision i It is the precision of category i, which indicates the proportion of samples predicted to be positive that are actually positive: Precision i =TP i +FP i i;

[0252] Recall i is the recall rate of category i, which indicates the proportion of samples that are correctly predicted as positive in all samples that are actually positive: i =TP i +FN i ;

[0253] This formula is weighted based on the traditional F1-score, by introducing the weight w i To balance the importance of different categories, especially in the case of dealing with class imbalance, it can better reflect the overall performance of the model for all categories.

[0254] The category-sensitive recall formula is as follows:

[0255]

[0256] In this formula, C is the total number of categories; D i is the number of samples of category i, which is used to balance the imbalance of category samples; TP i is the true class number of class i; FN i is the number of false negatives for category i. This formula introduces the inverse value of the number of class samples based on the recall rate As a weight, this makes the influence of minority classes be amplified when calculating the recall rate, thereby improving the model's ability to recognize minority classes.

[0257] The performance of the model was comprehensively analyzed using the above evaluation formula, and the results were compared with traditional methods. The following table summarizes the evaluation results:

[0258] index Traditional model results Improving model results Accuracy 0.85 0.89 Weighted Precision 0.83 0.88 Recall 0.80 0.86 Category-sensitive recall 0.78 0.84 F1-score 0.82 0.87 Generalized F1-score 0.81 0.88

[0259] As can be seen from the table above, the improved model outperforms the traditional model in all evaluation indicators. In particular, by introducing the weighted and category-sensitive improved formula, the model performs better in dealing with category imbalance, especially in terms of weighted precision and generalized F1-score. This proves the innovative design of the evaluation indicators of this patent, which effectively enhances the robustness and accuracy of the model in practical applications.

[0260] Furthermore, after the model evaluation is completed, the model parameters may need to be further tuned based on the evaluation results. The goal of tuning is to resolve the classification bias of the model on specific complaint types and ensure that the model performs evenly across all categories.

[0261] The tuning process includes:

[0262] Adjust hyperparameters: Adjust the model’s learning rate, weight decay coefficient and other hyperparameters based on the evaluation results to improve the overall performance of the model.

[0263] Improve model architecture: Adjust the model’s structural parameters such as the number of layers, number of neurons, activation function, etc. according to the classification performance.

[0264] Sample rebalancing: When the evaluation finds that the classification performance of some categories is poor, the performance of these categories can be improved by adjusting the data balancing strategy (such as SMOTE or weighted loss function).

[0265] Through continuous tuning, we ensure that the model reaches optimal performance before actual deployment, so that it can perform well in various types of complaint classification. Therefore, by using multi-index evaluation and cross-validation, the present invention ensures the stability and generalization ability of the model in different application scenarios, reduces the risk of overfitting of the model, and makes it more reliable in practical applications.

[0266] Step S50: deploy the trained multi-task learning model to the actual complaint handling system, receive complaint data, implement automatic classification and real-time severity scoring, and continuously optimize the system based on the feedback mechanism.

[0267] Specifically, step S50 includes steps S51-S53.

[0268] Step S51: The trained multi-task learning model based on improved NLP (Natural Language Processing) will be integrated into the complaint handling system.

[0269] The system will receive complaint data submitted by users (including structured data, text data, and voice data) and input these data into the NLP model for processing. The deployment steps are as follows:

[0270] Data preprocessing: The system automatically performs preprocessing steps such as data cleaning, word segmentation, and word embedding to ensure that the data format is consistent with that used when training the model.

[0271] Model inference: The preprocessed data will be input into the NLP model, and the model will output the classification results (such as "leakage", "crack", etc.) and severity score of each complaint.

[0272] Result display: The system displays the output results of the model to relevant processing personnel in a user-friendly manner or directly feeds back to the user so that appropriate processing measures can be taken in a timely manner.

[0273] Step S52: Real-time classification and feedback.

[0274] In actual deployment, the system will classify and score each newly submitted complaint data in real time. This process will ensure that complaints can be prioritized according to their urgency and type, improving the efficiency and accuracy of complaint handling.

[0275] Automatic classification: The system automatically assigns complaints to the appropriate department based on the model’s classification results. For example, a power supply issue may be immediately transferred to the power maintenance team.

[0276] Severity score: Based on the model's score, the system will mark high-severity complaints (such as issues involving structural safety) as priority, triggering an emergency processing process.

[0277] Feedback mechanism: The system will continuously collect feedback from users and processors, such as the accuracy of classification results, the effectiveness of processing results, etc. These feedbacks will be used to adjust and optimize the performance of the model to ensure the long-term adaptability of the model.

[0278] Step S53: Dynamic learning and optimization.

[0279] In order to maintain the efficiency and accuracy of the model, the system will regularly retrain and optimize the model. By adding new complaint data (including user feedback, actual processing results, etc.) to the training data set, the model will continue to learn and maintain a high level of performance in complaint classification and scoring.

[0280] Data collection: The system automatically collects and annotates new complaint data and processing feedback.

[0281] Model retraining: Use new data sets to regularly retrain the model and update model parameters to adapt to new complaint patterns.

[0282] Model update: Deploy the updated model to the system to replace the old model, ensuring that the system always uses the latest algorithms and parameters.

[0283] Furthermore, the technical solution of the present invention will be described in detail below through a specific embodiment of actual operation (recorded and available for reference).

[0284] On August 20, 2024, the effectiveness of the automatic classification system for housing quality and safety complaints based on improved NLP was verified at 14:00 in the afternoon in a residential building in area B. The outside temperature was 30 degrees Celsius, the humidity was 60%, and the air quality index was 100, which was slightly polluted. The property management system has received multiple housing quality complaints, and the system has started the automatic classification and severity scoring functions. The system collects and processes complaint data in real time through its built-in multimodal neural network.

[0285] Application in the above embodiments:

[0286] 14:02: The system extracts five housing quality complaint data from the property management system, which are: water leakage, wall cracks, power supply problems, floor subsidence, and unsealed doors and windows. The system first preprocesses these data, including cleaning outliers, processing missing data, and category coding.

[0287] 14:05: Through the multi-task learning model based on improved NLP, the system extracted features and enhanced semantics of text complaints. Combined with the knowledge of house structure, the system further analyzed the complaint text through the knowledge graph and generated preliminary classification results. The classification results are: one for each of water leakage, cracks, and power supply problems, and two for floor subsidence.

[0288] 14:08: Based on the multi-model embedding fusion processing of the complaint text, the system uses a multi-task learning model to score the severity of each complaint. The scoring results are:

[0289] Water leakage: Severity score 85,

[0290] Wall cracks: Severity score 70,

[0291] Power supply problem: Severity score 60,

[0292] Floor subsidence (two): severity score is 75 for both;

[0293] 14:10: The system generated priority treatment recommendations based on severity scores, listing complaints about water leaks and floor subsidence as priority, and complaints about power supply problems and wall cracks as secondary priorities.

[0294] 14:12: The system verifies and optimizes the classification model for various types of data. Based on the actual feedback results, the system updates the severity prediction model in real time through the adaptive attention mechanism and uploads the latest classification data to the property management system for further processing.

[0295] Comparative analysis:

[0296] In order to evaluate the performance of the system, the performance of the housing quality complaint handling method of the present invention and the traditional method was compared under the same environment. The specific data are shown in Table 1 below:

[0297] project Method of the present invention Traditional methods Classification Accuracy Initial: 80% Final: 90% Initial: 65% Final: 75% Severity score accuracy Initial: 75% Final: 85% Initial: 60% Final: 70% Complaint handling response time Average response time: 2 seconds Average response time: 5 seconds Model optimization frequency Dynamic optimization once every hour Fixed optimization every 24 hours System processing efficiency 20 complaints are handled every hour 4 complaints handled per hour

[0298] It can be seen from Table 1 that the method of the present invention is significantly superior to the traditional method in classification accuracy, severity scoring accuracy and response time, and significantly improves the efficiency of complaint handling through multi-task learning model and adaptive optimization mechanism.

[0299] In summary, the scheme of the present invention proposes for the first time a system deployment method of dynamic learning and adaptive feedback mechanism, which effectively solves the problems of data security and privacy protection. During the system deployment process, complaints are prioritized through automatic classification and severity scoring, and the system can continuously collect feedback information from users and processing personnel to achieve dynamic learning and model retraining, ensuring the efficiency and accuracy of the system in practical applications. At the same time, this scheme introduces strict data security measures in the handling of housing quality complaints for the first time to ensure the privacy and security of complaint data and avoid the data leakage risks existing in the prior art. Through the feedback mechanism, new complaint data and processing results are continuously collected, and the model is regularly retrained and optimized, so that the system can continuously adapt to new complaint types and changing processing requirements, and maintain long-term efficiency and adaptability.

[0300] Embodiment 2

[0301] The present invention provides an automatic classification device 100 for housing quality and safety complaints, which is used to automatically process complaint data related to housing quality and safety, and introduces a dynamic adaptive attention mechanism and a multi-task learning model technology, thereby providing a more efficient, intelligent and safe complaint processing solution. It should be noted that the implementation principle and specific implementation method of the automatic classification device 100 for housing quality and safety complaints can refer to the above-mentioned automatic classification method for housing quality and safety complaints, and will not be repeated below.

[0302] like Figure 6 As shown, the housing quality and safety complaint automatic classification device 100 includes:

[0303] The data acquisition module 10 is used to acquire housing quality and safety complaint data from the property management system, the online complaint platform, emails and telephone records, and use a preset multimodal neural network to uniformly convert the housing quality and safety complaint data into tabular data; wherein the tabular data includes complaint ID, complaint type, complaint time and location, complaint content summary, processing time, complaint severity score and processing status;

[0304] The model building module 20 is used to perform feature extraction and semantic enhancement on the table data, and to build a multi-task learning model combined with a dynamic adaptive attention mechanism; wherein the multi-task learning model includes a shared encoder and a classification task output head and a regression task output head; the classification task output head is used to map the hidden layer representation processed by the encoder to the probability distribution of various types of complaints, so as to be responsible for determining the specific category of the complaint; the regression task output head is used to map the hidden layer representation processed by the encoder to the output value of the regression task, so as to predict the severity or urgency score of the complaint;

[0305] The model training module 30 is used to obtain the data set required for multi-task learning model training, divide the data set into a training set, a validation set and a test set, and then train, validate and optimize the multi-task learning model;

[0306] A model evaluation module 40 is used to evaluate and verify the trained multi-task learning model, and further tune the parameters of the model that does not meet the requirements according to the evaluation results, so as to make the performance of the model balanced in all categories;

[0307] The model deployment module 50 is used to deploy the trained multi-task learning model to the actual complaint processing system, receive complaint data, realize automatic classification and real-time severity scoring, and continuously optimize the system based on the feedback mechanism.

[0308] Embodiment 3

[0309] The present invention provides a terminal, which includes a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, the various steps of the automatic classification method for housing quality and safety complaints as described in any of the above-mentioned implementation methods are implemented.

[0310] Embodiment 4

[0311] The present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various steps of the method for automatically classifying housing quality and safety complaints as described in any of the above-mentioned implementation methods.

[0312] Embodiment 5

[0313] The present invention provides a computer program product, including a computer program or instructions, which, when processed and executed, implement the various steps of the method for automatically classifying housing quality and safety complaints as described in any of the above-mentioned implementation modes.

[0314] In summary, the automatic classification method and device for housing quality and safety complaints provided by the present invention have the following features:

[0315] Beneficial effects:

[0316] (1) By introducing a dynamic adaptive attention mechanism and a multi-task learning model, the present invention can adjust the attention weight in real time according to the context of the complaint text, significantly improving the classification accuracy of complex, multi-topic complaint texts;

[0317] (2) Through feature extraction and semantic enhancement technology, the present invention effectively captures key information in the complaint text, especially the professional terms and implicit semantics in housing quality complaints, making the severity score more accurate and ensuring that high-priority issues can be handled in a timely manner;

[0318] (3) A multimodal neural network is used to process data from multiple sources such as property management systems, online complaint platforms, emails, and phone records, achieving unified optimization and integration of structured and unstructured data, providing high-quality input features for the model;

[0319] (4) Through the feedback mechanism, new complaint data and processing results are continuously collected, and the model is regularly retrained and optimized so that the system can continuously adapt to new complaint types and changing processing requirements, maintaining long-term efficiency and adaptability;

[0320] (5) By using multi-index evaluation and cross-validation, the present invention ensures the stability and generalization ability of the model in different application scenarios, reduces the risk of model overfitting, and makes it more reliable in practical applications.

[0321] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0322] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0323] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0324] In the embodiments provided by the present invention, it should be understood that the disclosed systems or devices / terminal equipment and methods can be implemented in other ways. For example, the system or device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system or unit can be electrical, mechanical or other forms.

[0325] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0326] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0327] The present invention is not limited to what is described in the specification and implementation modes, and therefore additional advantages and modifications can be easily realized by those skilled in the art. Therefore, without departing from the spirit and scope of the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein.

Claims

1. A method for automatically classifying housing quality and safety complaints, characterized in that: include: Step S10: Obtain housing quality and safety complaint data from the property management system, online complaint platform, emails and telephone records, and use a preset multimodal neural network to uniformly convert the housing quality and safety complaint data into tabular data; wherein the tabular data includes complaint ID, complaint type, complaint time and location, complaint content summary, processing time, complaint severity score and processing status; Step S20: performing feature extraction and semantic enhancement on the table data, and constructing a multi-task learning model combined with a dynamic adaptive attention mechanism; wherein the multi-task learning model includes a shared encoder and a classification task output head and a regression task output head; the classification task output head is used to map the hidden layer representation processed by the encoder to the probability distribution of various types of complaints, so as to be responsible for determining the specific category of the complaint; the regression task output head is used to map the hidden layer representation processed by the encoder to the output value of the regression task, so as to predict the severity or urgency score of the complaint; Step S30: obtaining a data set required for training the multi-task learning model, dividing the data set into a training set, a validation set and a test set, and then training, validating and optimizing the multi-task learning model; Step S40: performing model evaluation and verification on the trained multi-task learning model, and further tuning the parameters of the model that does not meet the requirements according to the evaluation results, so as to achieve balanced performance of the model in all categories; Step S50: deploy the trained multi-task learning model to the actual complaint handling system, receive complaint data, implement automatic classification and real-time severity scoring, and continuously optimize the system based on a feedback mechanism.

2. The automatic classification method for housing quality and safety complaints according to claim 1, characterized in that: The multimodal neural network includes a structured data processing subnetwork, a text processing subnetwork, a speech processing subnetwork and a feature fusion layer; Wherein, the step S10 includes: Step S11: performing structured data processing on the complaint data of the property management system, and inputting the obtained structured data into the structured data processing subnetwork; Step S12: performing unstructured data processing on the complaint data of the online complaint platform and the email, and inputting the word embedding vector and / or sentence embedding vector generated by the processing into the text processing sub-network; Step S13: performing voice data processing on the complaint data recorded in the telephone record to generate text output, converting the generated text data into a word embedding vector, and then inputting the generated word embedding vector into the voice processing subnetwork; Step S14: Integrate the features from the structured data, text data and voice data through the feature fusion layer to form a unified feature representation based on a table.

3. The automatic classification method for housing quality and safety complaints according to claim 2, characterized in that: The structured data processing subnetwork is composed of a series of fully connected layers, and performs feature extraction and processing on the structured data according to a first preset formula; the first preset formula is as follows: <h2 style=";text-align:left;direction:ltr">h1 = ReLU(W1x+b1) = h2 = ReLU(W2h1+b2) = y<h2 style=";text-align:left;direction:ltr"> structured <h2 style=";text-align:left;direction:ltr"> =W3h2+b3, Where x is the input structured data vector, W1, W2, W3 are weight matrices, b1, b2, b3 are biases, ReLU is an activation function used to introduce nonlinearity, and y structured is the output feature of the structured data processing subnetwork; The text processing sub-network is constructed based on the Transformer architecture and performs semantic understanding and feature extraction on the text according to the second preset formula; the second preset formula is as follows: In the formula, K represents the key matrix, which is extracted from the input word embedding or the output of the previous layer and is used to represent the features that need to be compared; Q represents the query matrix, which is also extracted from the input word embedding or the output of the previous layer and is used to find features related to the key matrix K; the key matrix K measures the similarity by performing a dot product with the query matrix Q; V represents the value matrix, which corresponds to the actual content of the input and generates the final output through weighted combination; Attention represents the output of the attention mechanism and is used to perform weighted averaging on the value matrix V; softmax represents a normalization function that converts the dot product result of the query matrix Q and the key matrix K into a probability distribution so that the sum of the attention weights is 1; is the scaled dot product of the query matrix Q and the key matrix K, d k is the dimension of the key vector, the scaling factor Used to avoid gradient vanishing due to large values; T is the matrix transpose; The speech processing subnetwork is used to first convert the speech data into text, and then perform corresponding data processing on the text through the second preset formula.

4. The automatic classification method for housing quality and safety complaints according to claim 3 is characterized by: The step S11 comprises: Step S111: extracting structured data containing detailed records of housing quality complaints from the property management system, including complaint ID, complaint type, complaint time and location, problem description, and processing status; Step S112: identifying and processing missing values ​​and outliers on the extracted data; Step S113: Encode the categorical data to obtain categorical features; Step S114: split the timestamp data into multiple time fields including "year, month, day, hour, minute", and perform periodic feature processing to capture the periodic features of time; Step S115: After completing the basic feature processing, different fields are combined in different ways to obtain composite features, which are then input into the structured data processing subnetwork to facilitate further feature extraction and fusion with other data sources; The step S12 comprises: Step S121: extracting original text data from the online complaint platform and emails, and cleaning the original text data; Step S122: performing word segmentation processing on the cleaned text to split the continuous text into independent words or sub-word units, and then converting the word segmented text into a word embedding vector; v word = Word2Vec(w), w is the input word, v word is the generated word embedding vector, which represents the numerical representation of word w in the embedding space; v is a high-dimensional vector; word is a subscript, indicating that this vector is associated with word w, and Word2Vec is the name of the preset model used to generate word embeddings; Step S123: for the entire sentence, the word embedding vectors contained therein are merged into a sentence embedding vector by averaging or weighted averaging; In the formula, v sentence is the generated sentence embedding vector, which represents the numerical representation of the entire sentence in the embedding space; sentence is a subscript, indicating that this vector is associated with the entire sentence; is a normalization factor used to calculate the average of all word embedding vectors in a sentence; is the embedding vector of the i-th word in the sentence; v is a vector representing the numerical representation of the i-th word; word i is a subscript indicating that the vector is associated with the i-th word in the sentence; N is the total number of words in the sentence; Step S124: inputting the generated word embedding and / or sentence embedding vector into the text processing subnetwork for further semantic understanding and feature extraction; The step S13 comprises: Step S131: extracting original voice data from the telephone record, and removing background noise from the original voice data to obtain an audio signal; Step S132: converting the audio signal into frequency domain features, and extracting higher-level speech features through Mel-frequency cepstral coefficients; Where, X k is the frequency domain energy, which indicates the energy at the kth frequency; K is the number of Mel filter banks; n is the index of the cepstral coefficient, which is used to represent different MFCC features; N is the number of MFCC features finally extracted; Step S133: Input the extracted frequency domain features or MFCC features into a preset speech recognition model to map the audio features into a phoneme sequence; wherein, for each time step, the speech recognition model calculates the probability p between the audio features and each phoneme: p(y t |X t )=softmax(W*h t +b), where X t is the audio feature, h t is the hidden state, W is the weight matrix, b is the bias term, y t is the predicted probability distribution of the phoneme; Step S134: decoding the phoneme sequence according to the context information to generate final text data to represent the actual content of the speech data; Step S135: According to the steps S122-S123, the final text data is converted into word embedding vectors and / or sentence embedding vectors, and input into the speech processing subnetwork for further semantic understanding and feature extraction.

5. The automatic classification method for housing quality and safety complaints according to claim 4, characterized in that: The step S14 comprises: Step S141: Integrate the features from structured data, text data and voice data by weighted averaging; the weighted averaging formula is as follows: y fused =α*y structured +β*y text +γ*y speech , where y fused It is the final feature vector after fusion, which represents the unified representation obtained by integrating the data features from different sources. It contains the comprehensive information of structured data, text data and voice data, and is used for subsequent classification and regression tasks. structured is the feature vector output from the structured data processing subnetwork, consisting of numerical and categorical data; text is the feature vector output from the text processing subnetwork, representing the semantic information of the complaint content; speech is the feature vector output from the speech processing subnetwork, which contains the speech features extracted from the phone records; α, β, and γ are weighting coefficients used to control the contribution of each data source to the final fusion feature, satisfying the following conditions: α+β+γ=1; Step S142: Map the fused high-dimensional features to a fixed output dimension and perform normalization processing, wherein the dimension of the fused feature vector is defined as d, and it is mapped to a fixed table format by the following formula: Output Table =W final *y fused +b final , where Output Table It is the final output tabular data, which contains all processed and fused features. Each row corresponds to a complaint record, and each column corresponds to a specific feature. fused is the fused feature vector obtained in the previous step, which integrates the feature information of structured data, text data and speech data. It is obtained by weighted fusion of the features output by different sub-networks and contains the comprehensive feature representation of all data sources; W final is the final weight matrix, which is used to transform the fused feature vector y fused Mapped to a fixed output dimension, responsible for converting high-dimensional fusion feature vectors into specific tabular data; b final is the final bias vector, which is related to W final Working together, they adjust the values ​​in the output table data so that the output does not depend entirely on the linear combination of the input features.

6. The automatic classification method for housing quality and safety complaints according to claim 5, characterized in that: The step S20 comprises: Step S21: extracting two text field data of "complaint content summary" and "voice record summary" from the table data, and then performing word segmentation processing on the extracted text field data to split the continuous text into independent words or sub-word units; Step S22: input the segmented text data into multiple pre-trained language models respectively, and each pre-trained language model generates a high-dimensional embedding representation of the corresponding text using its unique Transformer architecture; Step S23: nonlinearly fuse the high-dimensional embedding representations generated by different pre-trained language models according to a preset multi-model embedding fusion formula; the multi-model embedding fusion formula is as follows: In the formula, h fused represents the fused text embedding representation vector, which combines the features from different pre-language training models; α1, α2, α3 represent weighting coefficients to control the impact of high-dimensional embedding representations of different pre-trained language models on the final result; represents the text embedding vector generated by the pre-language training model; hRoBERTa represents the text embedding vector generated by the pre-language training model; h ALBERT represents the text embedding vector generated by the pre-language training model; e hRoBERTa represents the exponential operation of the text embedding vector to increase the model's nonlinear processing ability for numerical values; log(h ALBERT +1) indicates the logarithmic processing of the text embedding vector, which is used to smooth the data and compress the dynamic range; Step S24: fine-tune the fused embedding vector at the preset adaptive embedding layer node through a preset self-supervised learning formula to obtain a final embedding representation; wherein the self-supervised learning formula is as follows: In the formula, h adaptive represents the text embedding vector after adaptive adjustment to adapt to specific tasks through nonlinear transformation; W represents the weight matrix of the adaptive embedding layer, which is used for linear transformation of the embedding vector; b represents the bias vector of the adaptive layer; tanh represents the hyperbolic tangent activation function, which introduces nonlinearity and limits the output between [-1,1]; represents the square root of the embedding vector dimension, used to normalize the input; Step S25: extracting structured data from the table data, including numerical data and categorical data; For numerical data, normalization is used to scale its values ​​to between 0 and 1; for categorical data, label encoding or one-hot encoding is used to convert categorical information into numerical representation.

7. The automatic classification method for housing quality and safety complaints according to claim 6, characterized in that: The step S20 further includes: Step S26: query relevant knowledge from the external domain knowledge base through the preset knowledge graph query node, and combine the generated knowledge vector with the text embedding vector through the formula of the preset graph convolutional network node to generate a knowledge-enhanced text embedding vector; wherein the formula is as follows: In the formula, h knowledgeenhanced represents the text embedding vector after knowledge enhancement; W represents the weight matrix of the graph convolutional network node; GCN() represents the graph convolutional network operation, which is used to combine the adjacency matrix and the text embedding vector for convolution; A is the adjacency matrix of the knowledge graph, which represents the relationship between each node; deg(Δ) represents the degree matrix of the adjacency matrix A, which is used to normalize the adjacency matrix to ensure the stability of the convolution operation; b represents the bias vector to adjust the baseline value of the output; Step S27: The text embedding vector and the structured data feature are fused in a preset feature interaction network node through a preset logarithmic function to generate a more complex high-dimensional feature representation; wherein the logarithmic function is as follows: v interaction =log(h knowledgeenhanced ⊙v structured +1), In the formula, v interaction represents the feature vector after interaction, which combines the knowledge-enhanced text features and structured data features; h knowledgeenhanced represents the knowledge-enhanced text embedding vector; v structured Represents a structured data feature vector; ⊙ represents an element-wise multiplication operation, which is used to realize the element-by-element interaction between text features and structured data features; log() represents a logarithmic operation, which is used to smooth the interaction results and handle the dynamic range; Step S28: Process the fused feature vector through a preset self-attention mechanism node, and finally perform feature mapping at a preset fully connected layer node to generate a fixed-dimensional feature vector.

8. The automatic classification method for housing quality and safety complaints according to claim 7, characterized in that: The encoder is as follows: H = TransformerEncoder(v interaction , P), where H represents the hidden layer representation after being processed by the Transformer encoder, which contains rich semantic information extracted from the input features and is shared in the classification and regression tasks; the TransformerEncoder module processes the input feature representation through the self-attention mechanism and feedforward neural network to capture the relationship between features; v interaction represents the feature vector after interaction; P represents position encoding, which is used to maintain the position information of the input sequence to ensure that the model can correctly understand the features of each position in the sequence; The classification task output header is as follows: y class =softmax(W class ·H+b class ), where y class represents the output of the classification task, that is, the probability distribution of complaint categories. The probability of each category is calculated by the Softmax function to ensure that the sum of the probabilities of all categories is 1; W class is the weight matrix of the classification task output head, which is used to map the shared hidden layer representation H to the output space of the classification task; b class It is the bias item of the classification task output head, which is used to adjust the output result; The regression task output header is as follows: y reg =W reg ·H+b reg , where y reg is the output of the regression task, i.e., the complaint severity or urgency score; W reg It is the weight matrix of the regression task output head, which is used to map the shared hidden layer representation H to the output value of the regression task; b reg It is the bias term of the regression task output head, which is used to linearly adjust the output result; The joint loss function for optimizing the classification task and regression task in the multi-task learning process is as follows: In the formula, It is the total loss function of multi-task learning to measure the overall performance of the model on classification and regression tasks; is the loss function of the classification task, which is used to measure the matching degree between the model's predicted probability of each category and the actual category; C is the number of categories, y i is an indicator variable of the actual category; is the loss function of the regression task, which is used to measure the difference between the model prediction value and the actual value. N is the number of samples. is the actual severity score; class and λ reg are the loss weights of the classification task and the regression task, respectively, which are used to balance the impact of the two tasks in the total loss; The formula for dynamically adjusting the attention weight according to the context of the input text in the dynamic adaptive attention mechanism is as follows: In the formula, is the dynamic attention weight of the i-th position in the input sequence at time step t, which is calculated by the softmax function and changes with the content and position of the input text; Q t is the query matrix at time t, indicating the focus of the model on the text at the current time step; is the key matrix at the i-th position in the input sequence at time t, which is matched with the query matrix to find the most relevant information; is the value matrix of the i-th position in the input sequence at time t, containing the actual text information; n is the length of the input sequence, that is, the number of words in the text; d k is the dimension of the key matrix, which is used for normalization to prevent the gradient from disappearing due to excessive values.

9. The automatic classification method for housing quality and safety complaints according to claim 1, characterized in that: The step S30 comprises: Step S31: In the data set, new synthetic samples are generated by SMOTE technology to balance the number of minority classes, so as to generate new synthetic samples between minority class samples to increase the number of minority classes; wherein the formula for generating new samples is: x new =x i +δ·(x j -x i ), where x new is the generated synthetic sample, x i is a randomly selected minority class sample, x j From x i The sample is selected from the nearest neighbor samples of , δ is a random number between 0 and 1 to control the interpolation ratio between samples; Step S32: introducing respective weights for each category in the cross-entropy-based loss function so that the model pays more attention to the minority category during training; wherein the weighted loss function is: In the formula, is the weighted loss function, C is the total number of categories, and w i is the weight of category i, y i is the actual class label, is the probability of the category predicted by the model; Step S33: During the model training process, the parameters of the model are trained by using the adaptive learning rate characteristics of the preset Adam optimization algorithm and introducing a weight decay mechanism; wherein the model parameters θ t The update formula in the tth iteration is: In the formula, θ t is the model parameter after the tth iteration; η is the learning rate to control the step size of each parameter update; is the adaptive first-order moment estimate of the gradient, is the adaptive second-order moment estimate of the gradient; ε is a smoothing term to prevent division by zero errors; λ is a weight decay coefficient used to control the complexity of the model to reduce overfitting.

10. An automatic classification device for housing quality and safety complaints, characterized in that: include: A data acquisition module is used to acquire housing quality and safety complaint data from the property management system, online complaint platform, emails and telephone records, and use a preset multimodal neural network to uniformly convert the housing quality and safety complaint data into tabular data; wherein the tabular data includes complaint ID, complaint type, complaint time and location, complaint content summary, processing time, complaint severity score and processing status; A model building module, used to perform feature extraction and semantic enhancement on the table data, and to build a multi-task learning model combined with a dynamic adaptive attention mechanism; wherein the multi-task learning model includes a shared encoder and a classification task output head and a regression task output head; the classification task output head is used to map the hidden layer representation processed by the encoder to the probability distribution of various types of complaints, so as to be responsible for determining the specific category of the complaint; the regression task output head is used to map the hidden layer representation processed by the encoder to the output value of the regression task, so as to predict the severity or urgency score of the complaint; A model training module is used to obtain the data set required for training the multi-task learning model, divide the data set into a training set, a validation set and a test set, and then train, validate and optimize the multi-task learning model; A model evaluation module is used to evaluate and verify the trained multi-task learning model, and further tune the parameters of the model that does not meet the requirements based on the evaluation results, so that the performance of the model is balanced across all categories; The model deployment module is used to deploy the trained multi-task learning model into the actual complaint processing system, receive complaint data, realize automatic classification and real-time severity scoring, and continuously optimize the system based on the feedback mechanism.

Images