A method for evaluating social utility of a project based on a multi-modal large model Bert-Resnet

Abstract

The application discloses a kind of based on multi-modal big model Bert-Resnet's evaluation project social utility method, it is related to project evaluation technical field, it is difficult to understand the technical problem that semantic association behind text and image is solved, simultaneously, also lack self-attention mechanism, it is easy to cause sentiment analysis error, in turn, affect subsequent social utility evaluation result;Through multi-modal big model framework construction, two different types of data of text and image are integrated, multi-modal sentiment analysis is realized by BERT-ResNet model, which can more comprehensively capture the public's emotional attitude than single mode analysis. Through pre-training on public data set and fine-tuning on own data set, the problem of insufficient data is effectively overcome, and the generalization ability of the model is improved. Innovatively, the sentiment analysis result is converted into specific social utility indicators, providing a quantitative means for project social impact assessment.

Description

Technical Field

[0001] The present invention belongs to the field of project evaluation, and specifically provides a method for evaluating the social utility of a project based on a multimodal large model Bert-Resnet. Background Art

[0002] As the complexity and investment scale of social projects (such as public services, infrastructure construction, and environmental protection projects) continue to expand, social utility assessment has become a key tool for measuring whether projects achieve their intended social goals. The types of data carriers that information relies on in current online environments include text, images, and mixed multimodal formats. This has spurred research on sentiment analysis in different modalities. Text sentiment analysis, a relatively mature research topic in academia and industry, has produced many mature and effective solutions. However, as netizens' expressions change, multimodal data such as images, audio, and video are now present in daily life. Previous sentiment analysis solutions based on a single modality can no longer meet the needs of real-world applications. Driven by objective factors, research on sentiment analysis based on multimodal data has emerged.

[0003] Existing schemes for evaluating the social utility of projects generally use single-modal analysis, which makes it difficult to understand the semantic connections between text and images. At the same time, they lack a self-attention mechanism, which can easily cause sentiment analysis errors and affect the subsequent social utility evaluation results. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a method for evaluating the social utility of projects based on a multimodal large model Bert-Resnet, which is used to solve the technical problem that it is difficult to understand the semantic associations behind texts and images. At the same time, there is a lack of self-attention mechanism, which easily causes sentiment analysis errors and thus affects the subsequent social utility evaluation results.

[0005] To solve the above problems, the first aspect of the present invention provides a method for evaluating the social utility of a project based on a multimodal large model Bert-Resnet, comprising the following steps:

[0006] Based on the project's geographic location, the information collection area is divided into zones based on their distance from the project's geographic location. Zone labels are set based on their distance from the project's geographic location, and keywords are set for the corresponding zones. Text and image data related to the target project in social media are collected through a crawler system.

[0007] Pre-process the collected text and image data, and add area labels and project labels to the project data;

[0008] Analyze the relevance of the project data based on the project data area labels and project labels, as well as the amount of associated text and image data collected in the corresponding area. Based on the analysis results, divide the data collected in the area into groups with different relevance.

[0009] A BERT-ResNet multimodal fusion model is constructed to interactively fuse text feature vectors and image feature vectors through a self-attention mechanism. The fusion results of the data grouped by different relevance regions are passed to different fully connected layers for sentiment classification.

[0010] A transfer learning strategy was used to fine-tune and optimize the constructed BERT-ResNet multimodal fusion model, and the optimized model was used to perform sentiment classification probability analysis.

[0011] The social utility score is calculated based on the sentiment classification probability output by the model, and a visual evaluation report is generated.

[0012] Optionally, in an example of the above aspect, setting an area label according to the distance between the area and the project's geographical location and setting a keyword for the corresponding area includes the following steps:

[0013] According to the distance between the area and the project's geographic location, add labels to the data of different areas and record the corresponding distance data;

[0014] The keyword relevance coefficient Sc of the area is set by the following formula:

[0015]

[0016] Where dmax is the maximum distance between the edge of the information collection area and the project location, d is the average distance between the points on the corresponding area and the project location, k is the distance coefficient, which is set to 1% of the reciprocal of dmax, and cosθ is the cosine similarity between the search text phrase and the search keyword;

[0017] Extract keywords from the project text using the TF-IDF semantic algorithm as initial keywords;

[0018] Calculate the keyword relevance coefficient for each area. Based on a preset threshold, search the vocabulary for words with a keyword relevance coefficient greater than the threshold as synonyms, and use them together with the initial keywords as the crawling keywords for the area.

[0019] According to the crawling keywords set for the corresponding area, the distributed crawler system collects text data associated with the target project in social media, as well as the image data corresponding to the text data.

[0020] Optionally, in an example of the above aspect, preprocessing the collected text and image data and adding area labels and project labels to the project data includes the following steps:

[0021] Preprocess the collected text data, including data cleaning, removing irrelevant characters, using Word2Vec for word segmentation, and converting to the format required by the BERT model;

[0022] Preprocess the image data corresponding to the text data, including: converting it to a uniform image size, normalizing it, and converting it into a vector using the Ima2Vec semantic model to the feature vector required by the RESNET model;

[0023] Use the captured keywords set for the corresponding area as project labels, and add area labels and project labels to the preprocessed data.

[0024] Optionally, in an example of the above aspect, analyzing the relevance of the areas, and dividing the data collected from the areas into groups of areas with different relevance according to the analysis results, includes the following steps:

[0025] Obtain the area labels and project labels of the project data, as well as the amount of associated text and image data collected in the corresponding area, and analyze the area relevance Gs=(1-d) / dmax+Lb / Lbmax+Lc / Lcmax, where d, Lb, and Lc are the average distance between the points on the corresponding area and the project location, the number of keywords in the project label, and the amount of associated text and image data collected, respectively. dmax, Lbmax, and Lcmax are the maximum values ​​of d, Lb, and Lc in the area, respectively.

[0026] According to the analysis results, different area correlation threshold ranges are set, and the data collected in the area are divided into high-correlation area grouping, medium-correlation area grouping and low-correlation area grouping.

[0027] Optionally, in an example of the above aspect, a BERT-ResNet multimodal fusion model is constructed to interactively fuse text feature vectors and image feature vectors through a self-attention mechanism, including the following steps:

[0028] Extract feature vectors of text data through the pre-trained BERT model;

[0029] Extract image data feature vectors through the pre-trained RESNET model;

[0030] Based on the BERT model and the ResNet model, a BERT-ResNet multimodal fusion model is constructed, which interactively fuses text feature vectors and image feature vectors through a self-attention mechanism, including:

[0031] A concatenation layer is set to project the feature vectors extracted by the BERT model and the RESNET model to a unified dimension, concatenate them into joint features according to the channel dimension, and expand them into a sequence form.

[0032] The spliced ​​feature sequence is passed through a multi-head self-attention layer, single-head attention analysis is performed, and then multi-head splicing is performed;

[0033] The outputs of the multi-head self-attention layer are integrated through a linear layer to obtain a fused feature vector.

[0034] Optionally, in an example of the above aspect, extracting a feature vector of text data using a pre-trained BERT model includes:

[0035] The embedding layer inputs a text sequence, converts the text into a sequence of word tokens through the BERT tokenizer, performs embedding transformation, and generates an embedding matrix; the embedding matrix is ​​input into a 12-layer Transformer encoder, and finally outputs a hidden state sequence; the pooling layer aggregates the hidden state sequence to generate a global feature vector.

[0036] Optionally, in an example of the above aspect, extracting an image feature vector through a pre-trained RESNET model includes:

[0037] The input image undergoes multi-layer convolution and residual operations to generate a high-order feature map;

[0038] Perform global average pooling, compress the generated high-order feature map into spatial dimensions, output a multi-dimensional feature vector, and retain the channel dimension semantic information.

[0039] Optionally, in an example of the above aspect, the fusion results of the differently associated slice grouping data are respectively passed to different fully connected layers for sentiment classification, including the following steps:

[0040] The fused feature vectors of high-correlation area grouping, medium-correlation area grouping, and low-correlation area grouping data are passed to different fully connected layers respectively;

[0041] The fully connected layer performs a linear transformation on the input fusion feature vector, and usually uses the Softmax function to convert the linearly transformed feature vector into a probability distribution output;

[0042] The probability distribution output by the fully connected layer represents the probability that the text belongs to each sentiment category, and the category with the highest probability is selected as the final sentiment classification result.

[0043] Optionally, in an example of the above aspect, a transfer learning strategy is used to fine-tune and optimize the constructed BERT-ResNet multimodal fusion model, including the following steps:

[0044] The constructed BERT-ResNet multimodal fusion model is pre-trained using the MVSA-multiple dataset;

[0045] The pre-trained model is fine-tuned through transfer learning. During the retraining process of the pre-trained model trained with historical project data, the embedding layer and the first 6 layers of BERT and the first 3 layers of ResNet are frozen, and the multimodal fusion layer and classification layer are trained. At the same time, a progressive unfreezing strategy is adopted, unfreezing 10% of the frozen layers after every 2 training cycles.

[0046] Optionally, in an example of the above aspect, calculating a social utility score based on the sentiment classification probability output by the model and generating a visual evaluation report includes the following steps:

[0047] Obtain the results of the sentiment classification probability analysis performed by the model, and count the positive sentiment probability Pposi and the negative sentiment probability Pnegi in the associated data of the i-th item in the area;

[0048] Based on the statistical probabilities of positive and negative emotions, confidence weighting is introduced to calculate the social utility score CSi. The social utility score CSi = Σ(wi*(Pposi-Pnegi)) / Σwi, where wi = 1 / (1+e^{-10|Pposi-Pnegi|}) is the emotional certainty weight of the associated data of the i-th target item in the area.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] This paper builds a large multimodal model framework, integrating two different types of data: text and images. Using the BERT-ResNet model, it implements multimodal sentiment analysis, which more comprehensively captures public sentiment than single-modal analysis. By pre-training on public datasets and fine-tuning on proprietary datasets, the model effectively overcomes the data shortage and improves its generalization capabilities.

[0051] This invention innovatively transforms sentiment analysis results into specific social utility indicators, providing a quantitative means for assessing a project's social impact. The self-attention mechanism effectively integrates the outputs of the BERT and ResNet models. During model training, some BERT and ResNet layers are dynamically adjusted based on the characteristics of the dataset. This flexible training strategy helps improve the model's performance on specific tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 Schematic diagram of the process of the present invention;

[0054] Figure 2 Schematic diagram of the BERT-ResNet multimodal fusion model constructed for the present invention;

[0055] Figure 3 Schematic diagram of the process of fine-tuning and optimizing the multimodal fusion model of the present invention. DETAILED DESCRIPTION

[0056] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0057] See also Figure 1-Figure 3 In a first aspect, an embodiment of the present invention provides a method for evaluating the social utility of a project based on a multimodal large model Bert-Resnet, comprising the following steps:

[0058] Based on the project's geographic location, the information collection area is divided into zones according to their distance from the project's geographic location. Zone labels are set based on their distance from the project's geographic location, and keywords are set for the corresponding zones. Text and image data related to the target project in social media are collected through a crawler system.

[0059] Pre-process the collected text and image data, and add area labels and project labels to the project data;

[0060] Analyze the relevance of the project data based on the project data area labels and project labels, as well as the amount of associated text and image data collected in the corresponding area. Based on the analysis results, divide the data collected in the area into groups with different relevance.

[0061] A BERT-ResNet multimodal fusion model is constructed to interactively fuse text feature vectors and image feature vectors through a self-attention mechanism. The fusion results of the data grouped by different relevance regions are passed to different fully connected layers for sentiment classification.

[0062] A transfer learning strategy was used to fine-tune and optimize the constructed BERT-ResNet multimodal fusion model, and the optimized model was used to perform sentiment classification probability analysis.

[0063] The social utility score is calculated based on the sentiment classification probability output by the model, and a visual evaluation report is generated.

[0064] Specifically, in this embodiment, GIS geo-fencing technology is used to divide the concentric circle areas. Dividing lines are set every 500-1000km radius to divide the information collection area into circular areas centered on the project's geographical location.

[0065] The areas are numbered according to the geographical distance between the areas and the projects, and area labels are set according to the average distance between the detection points in the areas and the project's geographical location, for example, 002-1250km. Keywords are set for the corresponding areas, and text and image data related to the target project in social media are collected through the crawler system.

[0066] By building a large multimodal model framework, integrating two different types of data, text and images, the BERT-ResNet model enables multimodal sentiment analysis, which more comprehensively captures public sentiment and attitudes than single-modal analysis. Pre-training on public datasets and fine-tuning on proprietary datasets effectively overcomes the problem of insufficient data and improves the model's generalization capabilities. The results of sentiment analysis are innovatively converted into specific social utility indicators, providing a quantitative means for assessing the project's social impact. The self-attention mechanism effectively merges the outputs of the BERT and ResNet models. During model training, some BERT and ResNet layers are dynamically adjusted based on the characteristics of the dataset. This flexible training strategy helps improve the model's performance on specific tasks.

[0067] In one embodiment of the present invention, setting a zone label according to the distance between the zone and the project's geographic location and setting a keyword corresponding to the zone includes the following steps:

[0068] According to the distance between the area and the project's geographic location, add labels to the data of different areas and record the corresponding distance data;

[0069] The keyword relevance coefficient Sc of the area is set by the following formula:

[0070]

[0071] Where dmax is the maximum distance between the edge of the information collection area and the project location, d is the average distance between the points on the corresponding area and the project location, k is the distance coefficient, which is set to 1% of the reciprocal of dmax, and cosθ is the cosine similarity between the search text phrase and the search keyword;

[0072] Extract keywords from the project text using the TF-IDF semantic algorithm as initial keywords;

[0073] Calculate the keyword relevance coefficient for each area. Based on a preset threshold, search the vocabulary for words with a keyword relevance coefficient greater than the threshold as synonyms, and use them together with the initial keywords as the crawling keywords for the area.

[0074] According to the crawling keywords set for the corresponding area, the distributed crawler system collects text data associated with the target project in social media, as well as the image data corresponding to the text data.

[0075] In one embodiment of the present invention, preprocessing the collected text and image data and adding area labels and project labels to the project data includes the following steps:

[0076] Preprocess the collected text data, including data cleaning, removing irrelevant characters, using Word2Vec for word segmentation, and converting to the format required by the BERT model;

[0077] Preprocess the image data corresponding to the text data, including: converting it to a uniform image size, normalizing it, and converting it into a vector using the Ima2Vec semantic model to the feature vector required by the RESNET model;

[0078] Use the captured keywords set for the corresponding area as project labels, and add area labels and project labels to the preprocessed data.

[0079] In one embodiment of the present invention, analyzing the relevance of the regions and dividing the data collected from the regions into regions with different relevance groups according to the analysis results includes the following steps:

[0080] Obtain the area labels and project labels of the project data, as well as the amount of associated text and image data collected in the corresponding area, and analyze the area relevance Gs=(1-d) / dmax+Lb / Lbmax+Lc / Lcmax, where d, Lb, and Lc are the average distance between the points on the corresponding area and the project location, the number of keywords in the project label, and the amount of associated text and image data collected, respectively. dmax, Lbmax, and Lcmax are the maximum values ​​of d, Lb, and Lc in the area, respectively.

[0081] According to the analysis results, different area correlation threshold ranges are set, and the data collected in the area are divided into high-correlation area grouping, medium-correlation area grouping and low-correlation area grouping.

[0082] The data collected from areas with an area correlation Gs greater than 2.1 are classified as high-correlation areas, the data collected from areas with an area correlation Gs less than or equal to 2.1 and greater than 1.8 are classified as medium-correlation areas, and the data collected from areas with an area correlation Gs less than or equal to 1.8 are classified as low-correlation areas.

[0083] In one embodiment of the present invention, a BERT-ResNet multimodal fusion model is constructed to interactively fuse text feature vectors and image feature vectors through a self-attention mechanism, including the following steps:

[0084] Extract feature vectors of text data through the pre-trained BERT model;

[0085] Extract image data feature vectors through the pre-trained RESNET model;

[0086] Based on the BERT model and the ResNet model, a BERT-ResNet multimodal fusion model is constructed, which interactively fuses text feature vectors and image feature vectors through a self-attention mechanism, including:

[0087] A concatenation layer is set to project the feature vectors extracted by the BERT model and the RESNET model to a unified dimension, concatenate them into joint features according to the channel dimension, and expand them into a sequence form.

[0088] The spliced ​​feature sequence is passed through a multi-head self-attention layer, single-head attention analysis is performed, and then multi-head splicing is performed;

[0089] The outputs of the multi-head self-attention layer are integrated through a linear layer to obtain a fused feature vector.

[0090] Specifically, in this embodiment, the steps of the multimodal feature fusion method are as follows:

[0091] Perform feature preprocessing and splicing, including:

[0092] Set up the ‌Input Characteristics‌:

[0093] Text feature vector: h∈R dh , BERT output: dh=768;

[0094] Image feature vector: f∈R df , ResNet output: df=512;

[0095] ‌Dimension alignment: If you need to unify the dimensions, you can perform a linear projection on the image feature vector f:

[0096] f′=Wprojf+bproj,Wproj∈Rdh×df

[0097] After projection, f′∈Rdh , which is consistent with the dimension of the text feature vector h.

[0098] ‌Feature concatenation: Concatenate h and f (or f') along the channel dimension to form a joint feature sequence: S = [h; f] ∈ R dh+df or S = [h; f'] ∈ R 2dh ;

[0099] ‌Sequence expansion: Expand S into a sequence form to adapt the attention mechanism (if multiple position interactions need to be processed): X = [S; S] ∈ R 2djoint , (djoint = dh + df or 2dh).

[0100] Perform multi-head self-attention calculation, including:

[0101] ‌Parameter definition: number of heads nheads = 8; single head dimension dhead = djoint / nheads.

[0102] Perform linear transformation to generate query (Q), key (K), and value (V) matrices for joint features X: Q = XWQ, K = XWK, V = XWV; where WQ, WK, WV ∈ R djoint×djoint , are learnable parameters.

[0103] ‌Split-head operation: Split Q / K / V along the dimension into n heads sub-matrices.

[0104] Compute single-head attention ;

[0105] Output Attentioni ∈ R L*dhead , which focuses on specific interaction patterns between text-image features.

[0106] ‌Multi-head concatenation: Concatenated = [Attention1; Attention2; …; Attention(nheads)]

[0107] Perform feature integration and output by projecting the multi-head attention output back to the original dimension through linear projection.

[0108] In one embodiment of the present application, a feature vector of text data is extracted through a pre-trained BERT model, including:

[0109] The embedding layer inputs the text sequence, converts the text into a word token sequence through the BERT tokenizer, performs embedding transformation to generate an embedding matrix; the embedding matrix is input into a 12-layer Transformer encoder to finally output a hidden state sequence; the pooling layer aggregates the hidden state sequence to generate a global feature vector.

[0110] In one embodiment of the present invention, extracting image feature vectors through a pre-trained RESNET model includes:

[0111] The input image undergoes multi-layer convolution and residual operations to generate a high-order feature map;

[0112] Perform global average pooling, compress the generated high-order feature map into spatial dimensions, output a multi-dimensional feature vector, and retain the channel dimension semantic information.

[0113] Specifically, in this embodiment, a ResNet-34 network structure is used, which includes 34 layers of depth (including convolutional layers and residual blocks), and its layer distribution is as follows:

[0114] ‌Input layer‌: 3-channel RGB image, size 224×224 (needs preprocessing and alignment)

[0115] Convolutional layer: 7×7 convolution kernel (Stride=2), 64 output channels, followed by 3×3 max pooling (Stride=2)

[0116] ‌Residual block groups‌ (4 groups):

[0117] Block 1: 3×3×64 → 3×3×64 (repeated 3 times)

[0118] Block2: 3×3×128 → 3×3×128 (repeated 4 times)

[0119] Block3: 3×3×256 → 3×3×256 (repeated 6 times)

[0120] Block4: 3×3×512 → 3×3×512 (repeated 3 times) Each residual block contains a skip connection (SkipConnection)

[0121] The core parameters of the ResNet-34 network structure are set as follows:

[0122] Convolution kernel: All convolution layers use a uniform 3×3 size with a stride of 1 or 2 (adjusted according to the layer); Activation function: ReLU (Rectified Linear Unit); ReLU(x)=max(0,x)

[0123] Initialization: pre-trained weights (ImageNet dataset) are loaded; Optimizer: SGD (stochastic gradient descent), momentum factor 0.9, initial learning rate 0.1; Normalization: Batch Normalization (after each convolutional layer).

[0124] In one embodiment of the present invention, the fusion results of the differently associated patch grouping data are respectively passed to different fully connected layers for sentiment classification, including the following steps:

[0125] The fused feature vectors of high-correlation area grouping, medium-correlation area grouping, and low-correlation area grouping data are passed to different fully connected layers respectively;

[0126] The fully connected layer performs a linear transformation on the input fusion feature vector, and usually uses the Softmax function to convert the linearly transformed feature vector into a probability distribution output;

[0127] The probability distribution output by the fully connected layer represents the probability that the text belongs to each sentiment category, and the category with the highest probability is selected as the final sentiment classification result.

[0128] In one embodiment of the present invention, a transfer learning strategy is used to fine-tune and optimize the constructed BERT-ResNet multimodal fusion model, including the following steps:

[0129] The constructed BERT-ResNet multimodal fusion model is pre-trained using the MVSA-multiple dataset;

[0130] The pre-trained model is fine-tuned through transfer learning and trained using historical project data. During the retraining process, the embedding layer and the first 6 layers of BERT and the first 3 layers of ResNet are frozen, and the multimodal fusion layer and classification layer are trained. At the same time, a progressive unfreezing strategy is adopted, unfreezing 10% of the frozen layers after every 2 training cycles.

[0131] In one embodiment of the present invention, calculating a social utility score based on the sentiment classification probability output by the model and generating a visual evaluation report includes the following steps:

[0132] Obtain the results of the model's emotional classification probability analysis, and count the positive emotion probability Pposi and negative emotion probability Pnegi in the associated data of the i-th item in the area;

[0133] Based on the statistical probabilities of positive and negative emotions, confidence weighting is introduced to calculate the social utility score CSi. The social utility score CSi = Σ(wi*(Pposi-Pnegi)) / Σwi, where wi = 1 / (1+e^{-10|Pposi-Pnegi|}) is the emotional certainty weight of the associated data of the i-th target item in the area.

[0134] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for evaluating the social utility of a project based on a multimodal large model Bert-Resnet, characterized in that: The following steps are involved: Based on the project's geographic location, the information collection area is divided into zones based on their distance from the project's location. GIS geo-fencing technology is used to divide the zones into concentric circles. Zone labels are set based on their distance from the project's location, and keywords are assigned to the zones. A crawler system is used to collect text and image data related to the target project from social media. Preprocess the collected text and image data, and add area labels and project labels to the project data; analyze the area relevance based on the area labels and project labels of the project data, as well as the amount of associated text and image data collected in the corresponding area, and divide the data collected in the area into area groups with different relevance based on the analysis results; Construct a BERT-ResNet multimodal fusion model, interactively fuse text feature vectors and image feature vectors through a self-attention mechanism, and pass the fusion results of different correlation area grouping data to different fully connected layers for sentiment classification; the method of passing the fusion results of different correlation area grouping data to different fully connected layers for sentiment classification includes the following steps: passing the fusion feature vectors of high-correlation area grouping, medium-correlation area grouping, and low-correlation area grouping data to different fully connected layers respectively; the fully connected layer performs a linear transformation on the input fusion feature vector, and uses the Softmax function to convert the linearly transformed feature vector into a probability distribution output; the probability distribution output by the fully connected layer represents the probability of the text belonging to each sentiment category, and the category with the largest probability is selected as the final sentiment classification result; The constructed BERT-ResNet multimodal fusion model is fine-tuned and optimized using a transfer learning strategy, and sentiment classification probability analysis is performed using the optimized model; a social utility score is calculated based on the sentiment classification probability output by the model, and a visual evaluation report is generated. The social utility score is calculated based on the sentiment classification probability output by the model, and a visual evaluation report is generated, comprising the following steps: obtaining the result of sentiment classification probability analysis performed by the model, and statistically calculating the positive sentiment probability Pposi and the negative sentiment probability Pnegi in the associated data of the i-th target project in the area; introducing confidence weighting based on the statistically calculated probabilities of positive and negative sentiments, and calculating the social utility score CSi, where CSi = Σ(wi*(Pposi-Pnegi)) / Σwi, where is the sentiment certainty weight of the associated data of the i-th target item in the area.

2. The method for evaluating the social utility of a project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that: Set area labels based on the distance between the area and the project's geographic location, and set keywords for the corresponding areas. This includes the following steps: According to the distance between the area and the project's geographic location, add labels to the data of different areas and record the corresponding distance data; The keyword relevance coefficient Sc of the area is set by the following formula: Where dmax is the maximum distance between the edge of the information collection area and the project location, d is the average distance between the points on the corresponding area and the project location, k is the distance coefficient, which is set to 1% of the reciprocal of dmax, and cosθ is the cosine similarity between the search text phrase and the search keyword; Extract keywords from the project text using the TF-IDF semantic algorithm as initial keywords; Calculate the keyword relevance coefficient for each area. Based on a preset threshold, search the vocabulary for words with a keyword relevance coefficient greater than the threshold as synonyms, and use them together with the initial keywords as the crawling keywords for the area. According to the crawling keywords set for the corresponding area, the distributed crawler system collects text data associated with the target project in social media, as well as the image data corresponding to the text data.

3. The method for evaluating the social utility of a project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that: Preprocess the collected text and image data and add area labels and project labels to the project data, including the following steps: Preprocess the collected text data, including data cleaning, removing irrelevant characters, using Word2Vec for word segmentation, and converting to the format required by the BERT model; Preprocess the image data corresponding to the text data, including: converting it to a uniform image size, normalizing it, and converting it into a vector using the Ima2Vec semantic model to the feature vector required by the RESNET model; Use the captured keywords set for the corresponding area as project labels, and add area labels and project labels to the preprocessed data.

4. The method for evaluating the social utility of a project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that: Analyze the relevance of the areas and, based on the analysis results, divide the data collected from the areas into groups with different relevance, including the following steps: Obtain the area label and project label of the project data, as well as the amount of associated text and image data collected in the corresponding area, and analyze the area relevance: Gs = (1-d) / dmax + Lb / Lbmax + Lc / Lcmax, where d, Lb, and Lc are the average distance between the points on the corresponding area and the project location, the number of keywords in the project label, and the amount of associated text and image data collected, respectively. dmax, Lbmax, and Lcmax are the maximum values ​​of d, Lb, and Lc in the area, respectively. According to the analysis results, different area correlation threshold ranges are set, and the data collected in the area are divided into high-correlation area grouping, medium-correlation area grouping and low-correlation area grouping.

5. The method for evaluating the social utility of a project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that: Constructing a BERT-ResNet multimodal fusion model to interactively fuse text feature vectors and image feature vectors through a self-attention mechanism includes the following steps: Extract feature vectors of text data through the pre-trained BERT model; Extract image data feature vectors through the pre-trained RESNET model; Based on the BERT model and the ResNet model, a BERT-ResNet multimodal fusion model is constructed, which interactively fuses text feature vectors and image feature vectors through a self-attention mechanism, including: A concatenation layer is set to project the feature vectors extracted by the BERT model and the RESNET model to a unified dimension, concatenate them into joint features according to the channel dimension, and expand them into a sequence form. The spliced ​​feature sequence is passed through a multi-head self-attention layer, single-head attention analysis is performed, and then multi-head splicing is performed; The outputs of the multi-head self-attention layer are integrated through a linear layer to obtain a fused feature vector.

6. The method for evaluating the social utility of a project based on the multimodal large model Bert-Resnet according to claim 5, characterized in that: Through the pre-trained BERT model, feature vectors of text data are extracted, including: The embedding layer inputs a text sequence, converts the text into a word token sequence through the BERT word segmenter, performs embedding transformation, and generates an embedding matrix; the embedding matrix is ​​input into the 12-layer Transformer encoder, and finally outputs a hidden state sequence; the pooling layer aggregates the hidden state sequence to generate a global feature vector.

7. The method for evaluating the social utility of a project based on the multimodal large model Bert-Resnet according to claim 5, characterized in that: Through the pre-trained RESNET model, the image feature vector is extracted, including: The input image undergoes multi-layer convolution and residual operations to generate a high-order feature map; Perform global average pooling, compress the generated high-order feature map into spatial dimensions, output a multi-dimensional feature vector, and retain the channel dimension semantic information.

8. The method for evaluating the social utility of a project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that: The transfer learning strategy is used to fine-tune and optimize the constructed BERT-ResNet multimodal fusion model, including the following steps: The constructed BERT-ResNet multimodal fusion model is pre-trained using the MVSA-multiple dataset; The pre-trained model is fine-tuned through transfer learning. During the retraining process of the pre-trained model trained with historical project data, the embedding layer and the first 6 layers of BERT and the first 3 layers of ResNet are frozen, and the multimodal fusion layer and classification layer are trained. At the same time, a progressive unfreezing strategy is adopted, unfreezing 10% of the frozen layers after every 2 training cycles.

Images