Project social utility evaluation method based on multi-modal large model Bert-Resnet

Abstract

The invention discloses a method for evaluating social utility of a project based on a multi-modal large model Bert-Resnet, relates to the technical field of project evaluation, and solves the technical problems that semantic association behind a text and an image is difficult to understand, meanwhile, a self-attention mechanism is also lacked, sentiment analysis errors are easily caused, and then a subsequent social utility evaluation result is influenced. Through multi-modal large model framework construction, two different types of data of texts and images are integrated, multi-modal sentiment analysis is realized through a BERT-ResNet model, and compared with single-modal analysis, the public sentiment attitude can be more comprehensively captured. Through pre-training on a public data set and fine tuning on an own data set, the problem of insufficient data volume is effectively overcome, and the generalization ability of the model is improved. The sentiment analysis result is innovatively converted into a specific social utility index, and a quantitative means is provided for social influence evaluation of a project.

Description

Technical Field

[0001] The present invention belongs to the field of project evaluation, and specifically relates to a method for evaluating the social utility of projects based on the multimodal large model Bert-Resnet. Background Art

[0002] With the continuous expansion of the complexity and investment scale of social projects (such as public services, infrastructure construction, environmental protection projects, etc.), social utility evaluation has become a key tool for measuring whether a project achieves its expected social goals. In the current network environment, the data carrier types to which information adheres include text, images, and various mixed modal organizational forms. This has given rise to research on sentiment analysis in different modal forms. As a relatively mature research topic in academia and industry, text sentiment analysis has already produced many mature and effective solutions. However, with the change in the expression methods of Internet users, various modal data such as pictures, audio, and video appear in daily life. The previous sentiment analysis solutions based on a single modality can no longer meet the needs of real application scenarios, and the research on sentiment analysis based on multimodal data has been spawned under the traction of objective factor requirements.

[0003] In existing solutions for evaluating the social utility of projects, single-modal analysis is generally adopted, which is difficult to understand the semantic associations behind text and images. At the same time, it lacks a self-attention mechanism, which is prone to sentiment analysis errors, thereby affecting the subsequent social utility evaluation results. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; for this purpose, the present invention proposes a method for evaluating the social utility of projects based on the multimodal large model Bert-Resnet, which is used to solve the technical problems of being difficult to understand the semantic associations behind text and images, lacking a self-attention mechanism at the same time, being prone to sentiment analysis errors, and thus affecting the subsequent social utility evaluation results.

[0005] To solve the above problems, the first aspect of the present invention provides a method for evaluating the social utility of projects based on the multimodal large model Bert-Resnet, including the following steps:

[0006] According to the geographical location of the project, divide the information collection area into sub-areas according to the distance from the geographical location of the project, set area labels according to the distance between the sub-area and the geographical location of the project, and set keywords for the corresponding sub-areas, and collect text and image data associated with the target project in social media through a crawler system;

[0007] Preprocess the collected text and image data, and add area labels and project labels to the project data;

[0008] Analyze the relevance of regions based on the region tags and project tags of project data, as well as the quantity of associated text and image data collected in the corresponding regions. According to the analysis results, classify the data collected in the regions into different relevance region groups;

[0009] Construct a BERT-ResNet multi-modal fusion model, interactively fuse the text feature vectors and image feature vectors through the self-attention mechanism, and respectively transmit the fusion results of the data in different relevance region groups to different fully connected layers for sentiment classification;

[0010] Adopt a transfer learning strategy to fine-tune and optimize the constructed BERT-ResNet multi-modal fusion model, and perform sentiment classification probability analysis through the optimized model;

[0011] Calculate the social utility score according to the sentiment classification probability output by the model, and generate a visual evaluation report.

[0012] Optionally, in an example of the above aspect, set the region tags according to the distance between the region and the project geographical location, and set the keywords for the corresponding region, including the following steps:

[0013] Add tags to the data in different regions according to the distance between the region and the project geographical location, and record the corresponding distance data;

[0014] The keyword relevance coefficient Sc of the region is set by the following formula:

[0015]

[0016] where dmax is the maximum distance between the edge of the information collection area and the project location, d is the average distance between the point on the corresponding region and the project location, k is the distance coefficient, set to 1% of the reciprocal of dmax, and cosθ is the cosine similarity between the retrieved text phrase and the retrieved keyword;

[0017] Extract the keywords of the project text through the TF-IDF semantic algorithm as the initial keywords;

[0018] Calculate the keyword relevance coefficient of each region, and according to the preset threshold, retrieve the words in the thesaurus whose keyword relevance coefficient is greater than the threshold as synonyms, and use them together with the initial keywords as the capture keywords of the region;

[0019] According to the capture keywords set for the corresponding region, collect the text data associated with the target project in the social media and the image data corresponding to the text data through the distributed crawler system.

[0020] Optionally, in an example of the above aspect, preprocess the collected text and image data, and add area tags and project tags to the project data, including the following steps:

[0021] Preprocess the collected text data, including: data cleaning, removing irrelevant characters, tokenizing using Word2Vec, and converting to the format required by the BERT model;

[0022] Preprocess the image data corresponding to the text data, including: converting to a unified image size, performing normalization, performing vector conversion using the Ima2Vec semantic model, and converting to the feature vectors required by the RESNET model;

[0023] Use the scraping keywords set for the corresponding area as project tags, and add area tags and project tags to the preprocessed data.

[0024] Optionally, in an example of the above aspect, analyze the area correlation. According to the analysis results, divide the data collected in the area into different correlation area groups, including the following steps:

[0025] Obtain the area tags and project tags of the project data, as well as the quantities of the associated text and image data collected in the corresponding area, and analyze the area correlation Gs = (1 - d) / dmax + Lb / Lbmax + Lc / Lcmax, where d, Lb, and Lc are the average distance between the points in the corresponding area and the project location, the number of keywords in the project tags, and the quantity of the associated text and image data collected, respectively, and dmax, Lbmax, and Lcmax are the maximum values of d, Lb, and Lc in the area;

[0026] According to the analysis results, set different area correlation threshold ranges, and divide the data collected in the area into high-correlation area groups, medium-correlation area groups, and low-correlation area groups.

[0027] Optionally, in an example of the above aspect, construct a BERT-ResNet multimodal fusion model, and interactively fuse the text feature vectors and image feature vectors through the self-attention mechanism, including the following steps:

[0028] Extract the feature vectors of the text data through the pre-trained BERT model;

[0029] Extract the image data feature vectors through the pre-trained RESNET model;

[0030] Construct a BERT-ResNet multimodal fusion model based on the BERT model and the RESNET model, and interactively fuse the text feature vectors and image feature vectors through the self-attention mechanism, including:

[0031] Set up a splicing layer to project the feature vectors extracted by the BERT model and the RESNET model to a unified dimension, then splice them into joint features along the channel dimension and expand them into a sequence form;

[0032] Perform single-head attention analysis on the spliced feature sequence through a multi-head self-attention layer, and then perform multi-head splicing;

[0033] Integrate the output of the multi-head self-attention layer through a linear layer to obtain a fused feature vector.

[0034] Optionally, in an example of the above aspect, extract the feature vectors of text data through a pre-trained BERT model, including:

[0035] The embedding layer inputs a text sequence, converts the text into a sequence of word tokens through a BERT tokenizer, performs embedding transformation to generate an embedding matrix; the embedding matrix is input into a 12-layer Transformer encoder, and finally outputs a sequence of hidden states; the pooling layer aggregates the sequence of hidden states to generate a global feature vector.

[0036] Optionally, in an example of the above aspect, extract image feature vectors through a pre-trained RESNET model, including:

[0037] The input image undergoes multi-layer convolution and residual operations to generate a high-order feature map;

[0038] Perform global average pooling to compress the spatial dimension of the generated high-order feature map and output a multi-dimensional feature vector, retaining the semantic information of the channel dimension.

[0039] Optionally, in an example of the above aspect, respectively pass the fusion results of different relevance slice-grouped data to different fully connected layers for sentiment classification, including the following steps:

[0040] Respectively pass the fusion feature vectors of the high-relevance slice-grouped, medium-relevance slice-grouped, and low-relevance slice-grouped data to different fully connected layers;

[0041] The fully connected layer performs a linear transformation on the input fusion feature vector, and usually uses the Softmax function to convert the linearly transformed feature vector into a probability distribution for output;

[0042] The probability distribution output by the fully connected layer represents the probability that the text belongs to each sentiment category, and the category with the highest probability is selected as the result of the final sentiment classification.

[0043] Optionally, in an example of the above aspect, adopt a transfer learning strategy to fine-tune and optimize the constructed BERT-ResNet multi-modal fusion model, including the following steps:

[0044] Pre-train the constructed BERT-ResNet multi-modal fusion model through the MVSA-multiple dataset;

[0045] Fine-tune the pre-trained model through transfer learning. During the retraining process of training the pre-trained model with historical project data, freeze the embedding layer and the first 6 layers of BERT, as well as the first 3 layers of ResNet, train the multi-modal fusion layer and the classification layer. At the same time, adopt a progressive unfreezing strategy, unfreezing 10% of the frozen layers every 2 training cycles.

[0046] Optionally, in an example of the above aspect, calculate the social utility score according to the sentiment classification probability output by the model and generate a visual evaluation report, including the following steps:

[0047] Obtain the results of the model's sentiment classification probability analysis, and count the positive sentiment probability Pposi and negative sentiment probability Pnegi in the associated data of the i-th target project in the area;

[0048] According to the statistically positive and negative sentiment probabilities, introduce confidence weighting to calculate the social utility score CSi. The social utility score CSi = Σ(wi*(Pposi - Pnegi)) / Σwi, where wi = 1 / (1 + e^{-10|Pposi - Pnegi|}) is the sentiment certainty weight of the associated data of the i-th target project in the area.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] Through the construction of a multi-modal large model framework, the present invention integrates two different types of data, text and image, and realizes multi-modal sentiment analysis through the BERT-ResNet model, which can capture the public's sentiment attitude more comprehensively than single-modal analysis. By pre-training on a public dataset and fine-tuning on its own dataset, the problem of insufficient data volume is effectively overcome, and the generalization ability of the model is improved.

[0051] The present invention innovatively converts the sentiment analysis result into a specific social utility index, providing a quantitative means for the social impact assessment of projects. Through the self-attention mechanism, the outputs of the BERT and ResNet models are effectively fused. During the model training process, some layers of BERT and ResNet are dynamically adjusted according to the characteristics of the dataset. This flexible training strategy helps to improve the performance of the model on specific tasks. Description of the Drawings

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0053] Figure 1 Schematic diagram of the method flow of the present invention;

[0054] Figure 2 Schematic diagram of constructing the BERT-ResNet multi-modal fusion model of the present invention;

[0055] Figure 3 Schematic diagram of the method flow for fine-tuning and optimizing the multi-modal fusion model of the present invention. Specific embodiments

[0056] The following will clearly and completely describe the technical solutions of the present invention in combination with the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0057] Please refer to Figures 1 - 3 , an embodiment of the first aspect of the present invention provides a method for evaluating the social utility of a project based on the multi-modal large model Bert-Resnet, including the following steps:

[0058] According to the geographical location of the project, the information collection area is divided into sub-areas according to the distance from the geographical location of the project. Area labels are set according to the distance between the sub-area and the geographical location of the project, and keywords corresponding to the sub-area are set. Text and image data associated with the target project are collected through a crawler system.

[0059] Preprocess the collected text and image data, and add area labels and project labels to the project data.

[0060] According to the area labels and project labels of the project data, and the quantity of the associated text and image data collected in the corresponding area, analyze the area relevance. According to the analysis results, divide the data collected in the area into different relevance area groups.

[0061] Construct a BERT-ResNet multi-modal fusion model, interactively fuse the text feature vectors and image feature vectors through the self-attention mechanism, and respectively transmit the fusion results of the data in different relevance area groups to different fully connected layers for sentiment classification.

[0062] The constructed BERT-ResNet multi-modal fusion model is fine-tuned and optimized using a transfer learning strategy, and sentiment classification probability analysis is performed using the optimized model;

[0063] Calculate the social utility score based on the sentiment classification probability output by the model and generate a visual evaluation report.

[0064] Specifically, in this embodiment, the GIS geofencing technology is used to divide concentric circle areas. A dividing line is set every 500 - 1000 km radius range, and the information collection area is divided into circular areas centered on the geographical location of the project;

[0065] The areas are numbered according to the geographical distance between the area and the project, and area labels are set according to the average distance between the detection points in the area and the project's geographical location. For example, 002 - 1250 km, and keywords corresponding to the areas are set. The crawler system is used to collect text and image data related to the target project in social media.

[0066] Through the construction of a multi-modal large model framework, two different types of data, text and image, are integrated. Multi-modal sentiment analysis is realized through the BERT-ResNet model, which can capture the public's sentiment attitude more comprehensively than single-modal analysis. By pre-training on a public dataset and fine-tuning on a proprietary dataset, the problem of insufficient data volume is effectively overcome, and the generalization ability of the model is improved. Innovatively, the sentiment analysis results are transformed into specific social utility indicators, providing a quantitative means for the social impact assessment of the project. Through the self-attention mechanism, the outputs of the BERT and ResNet models are effectively fused. During the model training process, some layers of BERT and ResNet are dynamically adjusted according to the characteristics of the dataset. This flexible training strategy helps to improve the performance of the model on specific tasks.

[0067] In one embodiment of the present invention, area labels are set according to the distance between the area and the project's geographical location, and keywords corresponding to the areas are set, including the following steps:

[0068] Labels are added to the data of different areas according to the distance between the area and the project's geographical location, and the corresponding distance data is recorded;

[0069] The relevance coefficient Sc of the keywords of the area is set by the following formula:

[0070]

[0071] Where dmax is the maximum distance between the edge of the information collection area and the project location, d is the average distance between the points on the corresponding area and the project location, k is the distance coefficient, set to 1% of the reciprocal of dmax, and cosθ is the cosine similarity between the retrieved text phrase and the retrieved keyword;

[0072] Extract the keywords of the project text through the TF-IDF semantic algorithm as the initial keywords;

[0073] Calculate the keyword correlation coefficient of each area. According to the preset threshold, retrieve the words in the thesaurus whose keyword correlation coefficient is greater than the threshold as synonyms, and use them together with the initial keywords as the capture keywords for the area;

[0074] According to the set capture keywords for the corresponding area, collect the text data associated with the target project and the corresponding image data in the social media through the distributed crawler system.

[0075] In one embodiment of the present invention, preprocess the collected text and image data, and add area labels and project labels to the project data, including the following steps:

[0076] Preprocess the collected text data, including: data cleaning, removing irrelevant characters, performing word segmentation using Word2Vec, and converting it into the format required by the BERT model;

[0077] Preprocess the image data corresponding to the text data, including: converting to a unified image size, performing normalization processing, performing vector conversion using the Ima2Vec semantic model, and converting it into the feature vector required by the RESNET model;

[0078] Use the capture keywords set for the corresponding area as project labels, and add area labels and project labels to the preprocessed data.

[0079] In one embodiment of the present invention, analyze the area relevance. According to the analysis results, divide the data collected in the area into different relevance area groups, including the following steps:

[0080] Obtain the area labels and project labels of the project data, as well as the quantity of the associated text and image data collected in the corresponding area. Analyze the area relevance Gs = (1 - d) / dmax + Lb / Lbmax + Lc / Lcmax, where d, Lb, and Lc are the average distance between the points in the corresponding area and the project location, the number of keywords in the project label, and the quantity of the associated text and image data collected respectively, and dmax, Lbmax, and Lcmax are the maximum values of d, Lb, and Lc in the area;

[0081] According to the analysis results, set different area relevance threshold ranges, and divide the data collected in the area into high-relevance area groups, medium-relevance area groups, and low-relevance area groups.

[0082] The data collected from the areas with a regional correlation Gs greater than 2.1 are classified as high - correlation areas, the data collected from the areas with a regional correlation Gs less than or equal to 2.1 and greater than 1.8 are classified as medium - correlation areas, and the data collected from the areas with a regional correlation Gs less than or equal to 1.8 are classified as low - correlation areas.

[0083] In one embodiment of the present invention, a BERT - ResNet multi - modal fusion model is constructed, and the text feature vector and the image feature vector are interactively fused through a self - attention mechanism, including the following steps:

[0084] Extract the feature vector of the text data through a pre - trained BERT model;

[0085] Extract the image data feature vector through a pre - trained RESNET model;

[0086] Based on the BERT model and the RESNET model, a BERT - ResNet multi - modal fusion model is constructed, and the text feature vector and the image feature vector are interactively fused through a self - attention mechanism, including:

[0087] Set a splicing layer to project the feature vectors extracted by the BERT model and the RESNET model to a unified dimension, then splice them into a joint feature according to the channel dimension, and expand it into a sequence form;

[0088] Pass the spliced feature sequence through a multi - head self - attention layer, perform single - head attention analysis, and then perform multi - head splicing;

[0089] Integrate the output of the multi - head self - attention layer through a linear layer to obtain a fused feature vector.

[0090] Specifically, in this embodiment, the steps of the multi - modal feature fusion method are as follows:

[0091] Perform feature pre - processing and splicing, including:

[0092] Set the input features:

[0093] Text feature vector: h ∈ R dh , BERT output: dh = 768;

[0094] Image feature vector: f ∈ R df , ResNet output: df = 512;

[0095] Dimension alignment: If it is necessary to unify the dimension, a linear projection can be performed on the image feature vector f:

[0096] f′ = Wprojf + bproj, Wproj ∈ Rdh×df

[0097] After projection, f′ ∈ Rdh , which is consistent with the dimension of the text feature vector h.

[0098] Feature concatenation: Concatenate h and f (or f′) along the channel dimension to form a joint feature sequence: S = [h; f] ∈ R dh+df or S = [h; f′] ∈ R 2dh ;

[0099] Sequence expansion: To adapt to the attention mechanism, expand S into a sequence form (if multi-position interaction needs to be processed): X = [S; S] ∈ R 2djoint , (djoint = dh + df or 2dh);

[0100] Perform multi-head self-attention calculation, including:

[0101] Parameter definition: The number of heads nheads = 8; The dimension of a single head dhead = djoint / nheads;

[0102] Perform linear transformation to generate query (Q), key (K), and value (V) matrices for the joint feature X: Q = XWQ, K = XWK, V = XWV; where WQ, WK, WV ∈ R djoint×djoint , which are learnable parameters.

[0103] Split-head operation: Split Q / K / V along the dimension into n heads sub-matrices;

[0104] Calculate single-head attention ;

[0105] Output Attentioni ∈ R L*dhead , focusing on specific interaction patterns between text and image features.

[0106] Multi-head concatenation: Concatenated = [Attention1; Attention2; …; Attention(nheads)]

[0107] Perform feature integration and output, and map the multi-head attention output back to the original dimension through linear projection.

[0108] In one embodiment of the present invention, through a pre-trained BERT model, extract the feature vector of text data, including:

[0109] The embedding layer inputs the text sequence, converts the text into a sequence of word tokens through the BERT tokenizer, performs embedding transformation to generate an embedding matrix; inputs the embedding matrix into a 12-layer Transformer encoder, and finally outputs a sequence of hidden states; the pooling layer aggregates the sequence of hidden states to generate a global feature vector.

[0110] In one embodiment of the present invention, an image feature vector is extracted through a pre-trained RESNET model, including:

[0111] The input image undergoes multi-layer convolution and residual operations to generate a high-order feature map;

[0112] Global average pooling is performed to compress the spatial dimension of the generated high-order feature map and output a multi-dimensional feature vector, retaining the semantic information of the channel dimension.

[0113] Specifically, in this embodiment, the ResNet-34 network structure is adopted, which contains 34 layers of depth (including convolutional layers and residual blocks), and its hierarchical distribution is as follows:

[0114] Input layer: 3-channel RGB image, size 224×224 (requires preprocessing for alignment)

[0115] Convolutional layer: 7×7 convolutional kernel (Stride = 2), 64 output channels, followed by 3×3 max pooling (Stride = 2)

[0116] Residual block group (4 groups):

[0117] Block1: 3×3×64 → 3×3×64 (repeated 3 times)

[0118] Block2: 3×3×128 → 3×3×128 (repeated 4 times)

[0119] Block3: 3×3×256 → 3×3×256 (repeated 6 times)

[0120] Block4: 3×3×512 → 3×3×512 (repeated 3 times). Each residual block contains a skip connection.

[0121] The core parameters of the ResNet-34 network structure are set as follows:

[0122] Convolutional kernel: All convolutional layers uniformly adopt a 3×3 size, and the stride is 1 or 2 (adjusted according to the layer); Activation function: ReLU (Rectified Linear Unit); ReLU(x) = max(0, x)

[0123] Initialization: Load pre-trained weights (ImageNet dataset); Optimizer: SGD (Stochastic Gradient Descent), momentum factor 0.9, initial learning rate 0.1; Normalization: Batch Normalization (after each convolutional layer).

[0124] In one embodiment of the present invention, the fusion results of different relevance slice group data are respectively passed to different fully-connected layers for sentiment classification, including the following steps:

[0125] Respectively pass the fusion feature vectors of the high-relevance slice group, medium-relevance slice group, and low-relevance slice group data to different fully-connected layers;

[0126] The fully-connected layer performs a linear transformation on the input fusion feature vector, and usually uses the Softmax function to convert the linearly transformed feature vector into a probability distribution for output;

[0127] The probability distribution output by the fully-connected layer represents the probability that the text belongs to each sentiment category, and the category with the highest probability is selected as the result of the final sentiment classification.

[0128] In one embodiment of the present invention, a transfer learning strategy is adopted to fine-tune and optimize the constructed BERT-ResNet multi-modal fusion model, including the following steps:

[0129] Pre-train the constructed BERT-ResNet multi-modal fusion model through the MVSA-multiple dataset;

[0130] Fine-tune the pre-trained model through transfer learning, train the pre-trained model with historical project data, and during the training process, freeze the embedding layer and the first 6 layers of BERT, as well as the first 3 layers of ResNet, train the multi-modal fusion layer and the classification layer, and at the same time, adopt a progressive unfreezing strategy, unfreezing 10% of the frozen layers every 2 training cycles.

[0131] In one embodiment of the present invention, calculate the social utility score according to the sentiment classification probability output by the model and generate a visual evaluation report, including the following steps:

[0132] Obtain the results of the sentiment classification probability analysis in the model, and count the positive sentiment probability Pposi and negative sentiment probability Pnegi in the associated data of the i-th target project in the area;

[0133] According to the statistically positive sentiment probability and negative sentiment probability, introduce confidence weighting to calculate the social utility score CSi, where the social utility score CSi = Σ(wi*(Pposi - Pnegi)) / Σwi, and wi = 1 / (1 + e^{-10|Pposi - Pnegi|}) is the sentiment certainty weight of the associated data of the i-th target project in the area.

[0134] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for evaluating the social utility of an assessment project based on the multimodal large model Bert-Resnet, characterized in that, It includes the following steps: According to the geographical location of the project, divide the information collection area into sub - areas according to the distance from the geographical location of the project. Set sub - area labels according to the distance between the sub - area and the geographical location of the project, and set keywords for the corresponding sub - areas. Collect text and image data related to the target project in social media through a crawler system; Pre - process the collected text and image data, and add sub - area labels and project labels to the project data; According to the sub - area labels and project labels of the project data, as well as the quantity of the associated text and image data collected in the corresponding sub - areas, analyze the sub - area relevance. According to the analysis results, divide the data collected in the sub - areas into different relevance sub - area groups; Construct a BERT - ResNet multi - modal fusion model, interactively fuse the text feature vectors and image feature vectors through the self - attention mechanism, and respectively transmit the fusion results of the data in different relevance sub - area groups to different fully - connected layers for sentiment classification; Adopt a transfer learning strategy to fine - tune and optimize the constructed BERT - ResNet multi - modal fusion model, and perform sentiment classification probability analysis through the optimized model; Calculate the social utility score according to the sentiment classification probability output by the model, and generate a visual evaluation report.

2. The method for evaluating the social utility of an evaluation project based on the multimodal large model Bert-Resnet according to claim 1, wherein Set sub - area labels according to the distance between the sub - area and the geographical location of the project, and set keywords for the corresponding sub - areas, including the following steps: Add labels to the data of different sub - areas according to the distance between the sub - area and the geographical location of the project, and record the corresponding distance data; The keyword relevance coefficient Sc of the sub - area is set by the following formula: where dmax is the maximum distance between the edge of the information collection area and the project location, d is the average distance between the points on the corresponding sub - area and the project location, k is the distance coefficient, set to 1% of the reciprocal of dmax, and cosθ is the cosine similarity between the retrieved text phrase and the retrieved keyword; Extract keywords of the project text through the TF - IDF semantic algorithm as the initial keywords; Calculate the keyword relevance coefficients of each sub - area. According to a preset threshold, retrieve the words in the thesaurus whose keyword relevance coefficients are greater than this threshold as synonyms, and use them together with the initial keywords as the scraping keywords for the sub - area; According to the set scraping keywords for the corresponding sub - area, collect text data related to the target project in social media through a distributed crawler system, as well as the image data corresponding to the text data.

3. The method for evaluating the social utility of an evaluation project based on the multimodal large model Bert-Resnet according to claim 1, wherein, Pre - process the collected text and image data, and add sub - area labels and project labels to the project data, including the following steps: Pre - process the collected text data, including: data cleaning, removing irrelevant characters, performing word segmentation using Word2Vec, and converting it into the format required by the BERT model; Pre - process the image data corresponding to the text data, including: converting to a unified image size, performing normalization processing, using the Ima2Vec semantic model for vector conversion, and converting it into the feature vectors required by the RESNET model; Use the scraping keywords set for the corresponding sub - area as project labels, and add sub - area labels and project labels to the pre - processed data.

4. A method for evaluating the social utility of an assessment project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that, Analyze the relevance of the regions. According to the analysis results, divide the data collected from the regions into different relevance region groups, including the following steps: Obtain the region labels and project labels of the project data, as well as the quantity of the associated text and image data collected from the corresponding regions. Analyze the region relevance Gs = (1 - d) / dmax + Lb / Lbmax + Lc / Lcmax, where d, Lb, and Lc are respectively the average distance between the points on the corresponding region and the project location, the number of keywords in the project label, and the quantity of the associated text and image data collected, and dmax, Lbmax, and Lcmax are respectively the maximum values of d, Lb, and Lc in the region; According to the analysis results, set different region relevance threshold ranges, and divide the data collected from the regions into high-relevance region groups, medium-relevance region groups, and low-relevance region groups.

5. A method for evaluating the social utility of an evaluation project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that Construct a BERT-ResNet multimodal fusion model to interactively fuse the text feature vectors and image feature vectors through the self-attention mechanism, including the following steps: Extract the feature vectors of the text data through the pre-trained BERT model; Extract the image data feature vectors through the pre-trained RESNET model; Construct a BERT-ResNet multimodal fusion model based on the BERT model and the RESNET model, and interactively fuse the text feature vectors and image feature vectors through the self-attention mechanism, including: Set a splicing layer to project the feature vectors extracted by the BERT model and the RESNET model to a unified dimension, then splice them into a joint feature according to the channel dimension, and expand it into a sequence form; Perform single-head attention analysis on the spliced feature sequence through the multi-head self-attention layer, and then perform multi-head splicing; Integrate the output of the multi-head self-attention layer through a linear layer to obtain the fused feature vector.

6. The method for evaluating the social utility of an assessment project based on the multimodal large model Bert-Resnet according to claim 5, characterized in that, Extract the feature vectors of the text data through the pre-trained BERT model, including: The embedding layer inputs the text sequence, converts the text into a sequence of word tokens through the BERT tokenizer, performs embedding transformation to generate an embedding matrix; input the embedding matrix into the 12-layer Transformer encoder, and finally output the hidden state sequence; the pooling layer aggregates the hidden state sequence to generate a global feature vector.

7. The method for evaluating the social utility of an assessment project based on the multimodal large model Bert-Resnet according to claim 5, characterized in that Extract the image feature vectors through the pre-trained RESNET model, including: The input image undergoes multi-layer convolution and residual operations to generate a high-order feature map; Perform global average pooling to compress the spatial dimension of the generated high-order feature map, and output a multi-dimensional feature vector, retaining the semantic information of the channel dimension.

8. A method for evaluating the social utility of an evaluation project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that, Pass the fusion results of the data in different relevance region groups to different fully connected layers for sentiment classification, including the following steps: Respectively pass the fusion feature vectors of the high-relevance region group, medium-relevance region group, and low-relevance region group data to different fully connected layers; The fully connected layer performs a linear transformation on the input fusion feature vector, and usually uses the Softmax function to convert the linearly transformed feature vector into a probability distribution for output; The probability distribution output by the fully connected layer, representing the probability that the text belongs to each sentiment category, and the category with the highest probability is selected as the final sentiment classification result.

9. A method for evaluating the social utility of an evaluation project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that, Adopt a transfer learning strategy to fine-tune and optimize the constructed BERT-ResNet multimodal fusion model, including the following steps: Pre-train the constructed BERT-ResNet multimodal fusion model through the MVSA-multiple dataset; Fine-tune the pre-trained model through transfer learning. During the retraining process, freeze the embedding layer and the first 6 layers of BERT, as well as the first 3 layers of ResNet, and train the multimodal fusion layer and the classification layer. At the same time, adopt a progressive thawing strategy, and thaw 10% of the frozen layers every 2 training cycles.

10. A method for evaluating the social utility of an assessment project based on the multimodal large model Bert-Resnet according to claim 1, characterized in that, Calculate the social utility score based on the sentiment classification probability output by the model and generate a visual evaluation report, including the following steps: Obtain the results of the sentiment classification probability analysis of the model, and count the positive sentiment probability Pposi and the negative sentiment probability Pnegi in the associated data of the i-th target project in the area; According to the statistically positive and negative sentiment probabilities, introduce confidence weighting to calculate the social utility score CSi. The social utility score CSi = Σ(wi*(Pposi - Pnegi)) / Σwi, where wi = 1 / (1 + e^{-10|Pposi - Pnegi|}) is the sentiment certainty weight of the associated data of the i-th target project in the area.

Images