An e-commerce comment noise reduction method, device, medium and product

Abstract

The application discloses an e-commerce comment noise reduction method and device, medium and product, and relates to the field of data analysis. First, the original data of user comments is obtained, including user ID, comment ID, comment time, comment content (including comment text and comment pictures), IP attribution of the comment user and merchant information; a time decay factor is calculated according to the comment time; text processing and sentiment analysis are performed on the comment text to calculate a good / bad sentiment score; a picture-text correlation check is performed on the comment pictures and the comment text to calculate a picture-text score; a spatial correlation check is performed according to the comment pictures and the IP attribution of the comment user to calculate a geographic score; an industry coefficient is determined according to the merchant information; a credibility score is calculated according to the time decay factor, the good / bad sentiment score, the picture-text score, the geographic score and the industry coefficient; and valid / invalid comments are identified according to the credibility score, which greatly improves the efficiency and accuracy of valid / invalid comment identification.

Description

Technical Field

[0001] This application relates to the technical field of data analysis, and particularly to an e-commerce review noise reduction method, device, medium and product. Background Art

[0002] The e-commerce review noise reduction technology stems from the need for quality control of a large amount of user-generated content (UGC). With the number of reviews on e-commerce platforms (such as the Amazon platform) exceeding tens of billions, reviews have become the core reference for consumers' decisions, but the noise problem severely restricts their value. Specifically, the noise problem usually includes: (1) Noise diversity: including meaningless short reviews (such as "good"), mechanically repeated content (system default good reviews), false reviews (brushing orders), and malicious offensive language; (2) Semantic complexity: The emotional tendency of the same word varies significantly under different categories (such as "light" is positive for a suitcase but may be negative for a power bank); (3) Dynamic evolution: Cyber black and gray industries use homophonic words (such as "Weixin number" instead of "WeChat number") to avoid detection and are difficult to update in real time.

[0003] Existing e-commerce review noise reduction technologies mainly include traditional signal processing methods, rule- and dictionary-based methods, machine learning and deep learning technologies, etc. However, traditional signal processing methods rely on artificial experience parameters and have poor scene adaptability, such as being difficult to adapt to the dynamic changes of live outdoor noise. Rule- and dictionary-based methods can remove high-frequency meaningless words such as "de" and "shi", but cannot solve semantic noise. For example, "The express delivery is fast" is of no value for product quality analysis and is ineffective for rewritten brush reviews (such as false good reviews with adjusted word order). Machine learning and deep learning models are difficult to lightweight (such as the LSTM layer causing delays) and rely on the quality of labeled data. Summary of the Invention

[0004] The purpose of this application is to provide an e-commerce review noise reduction method, device, medium and product, which can improve the accuracy and efficiency of identifying effective / invalid reviews by integrating visual-semantic information for multi-modal joint noise reduction and through user behavior feature fusion.

[0005] To achieve the above purpose, the following solutions are provided in this application.

[0006] In the first aspect, this application provides an e-commerce review noise reduction method, including:

[0007] Obtain the original user review data, including user ID, review ID, review time, review content, the IP location of the reviewing user, and merchant information; the review content includes review text and review pictures;

[0008] Calculate the time decay factor according to the review time;

[0009] text processing and sentiment analysis on the review text to calculate a positive sentiment score and a negative sentiment score;

[0010] image-text correlation verification on the review image and the review text to calculate an image-text score;

[0011] spatial correlation verification according to the review image and the IP address of the review user to calculate a geographic score;

[0012] determining an industry coefficient according to the merchant information;

[0013] calculating a credibility score according to the time decay factor, the positive sentiment score, the negative sentiment score, the image-text score, the geographic score, and the industry coefficient;

[0014] identifying valid reviews and invalid reviews according to the credibility score.

[0015] Optionally, the time decay factor is calculated according to the review time, specifically including:

[0016] the time decay factor is calculated by the formula ; wherein is the review time; is the current time; is the industry decay rate.

[0017] Optionally, the text processing and sentiment analysis on the review text to calculate a positive sentiment score and a negative sentiment score specifically includes:

[0018] text segmentation is performed on the review text to remove irrelevant symbols and stop words, and a word list after text processing is obtained;

[0019] comment keywords and key sentiment words in the word list are screened out based on the part of speech or a dictionary, and the polarity and sentiment intensity score of each key sentiment word are determined; the polarity of the key sentiment word includes positive, negative, and neutral;

[0020] the positive sentiment score and the negative sentiment score are obtained by summarizing the polarity and sentiment intensity score of each key sentiment word.

[0021] Optionally, the image-text correlation verification on the review image and the review text to calculate an image-text score specifically includes:

[0022] visual feature extraction is performed on each review image to extract a corresponding image feature vector;

[0023] text feature extraction is performed on the review text to extract a text feature vector;

[0024] ​The image feature vector and the text feature vector are normalized, and the image-text similarity between each image feature vector and text feature vector is calculated after normalization;

[0025] When the image-text similarity is greater than the similarity threshold, it is determined that the corresponding comment picture is an image-text consistent comment picture;

[0026] The formula is used The image-text score is calculated ; wherein is the number of image-text consistent comment pictures; is the total number of comment pictures.

[0027] Optionally, the spatial correlation verification according to the comment picture and the IP belonging place of the comment user is performed to calculate the geographical score, specifically including:

[0028] The GPS position of the comment picture is extracted, and the distance between the GPS position of the comment picture and the center of the IP belonging place of the comment user is calculated ;

[0029] The formula is used The geographical score is calculated ; wherein represents taking the maximum value; represents the zero distance threshold.

[0030] Optionally, the credibility score is calculated according to the time decay factor, the positive review sentiment score, the negative review sentiment score, the image-text score, the geographical score and the industry coefficient, specifically including:

[0031] The formula is used The credibility score is calculated ; wherein represents the industry coefficient; represents the user historical credibility; represents the picture authenticity coefficient; represents the positive review sentiment score; represents the positive review weight proportion; represents the negative review sentiment score; represents the negative review weight proportion.

[0032] Optionally, the valid comment and the invalid comment are identified according to the credibility score, specifically including:

[0033] When the credibility score is lower than the score lower limit threshold, it is determined that the corresponding user comment is an invalid comment, and the invalid comment is folded or hidden; otherwise, it is determined that the corresponding user comment is a valid comment, and the valid comment is displayed from high to low according to the credibility score.

[0034] In a second aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the e-commerce comment denoising method.

[0035] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the e-commerce comment denoising method.

[0036] In a fourth aspect, the present application provides a computer program product comprising a computer program executable by a processor to implement the e-commerce comment denoising method.

[0037] According to the specific embodiments provided by the present application, the following technical effects are disclosed.

[0038] The e-commerce comment denoising method, device, medium and product provided by the present application can solve the key pain points in the traditional denoising technology through technical innovation, perform multi-modal joint denoising by integrating the visual-semantic information of the comment pictures and comment texts, and break through the limitation of single text analysis; through the verification of the correlation between the picture and the text and the spatial correlation, the false / invalid comments with inconsistent picture and text or inconsistent address can be effectively identified; the user behavior feature fusion is combined with the user behavior track (such as user ID, comment ID, comment time, comment content, IP attribution of the comment user and merchant information), the credibility score is calculated by weighting, and the identification accuracy and efficiency of the invalid / valid comments are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0040] Figure 1 FIG. 1 is a flowchart of the e-commerce comment denoising method of the present application;

[0041] Figure 2 FIG. 3 is a schematic diagram of the polarity and intensity division of the sentiment words;

[0042] Figure 3 FIG. 4 is a schematic diagram of the comment picture in the embodiment of the present application. DETAILED DESCRIPTION

[0043] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0044] The present application provides an e-commerce comment noise reduction method, device, medium and product, aiming to improve the effective / ineffective comment recognition accuracy and efficiency by integrating visual-semantic information for multi-modal joint noise reduction and by fusing user behavior characteristics.

[0045] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0046] In one exemplary embodiment, as shown in Figure 1 An e-commerce comment noise reduction method is provided, including the following steps 1 to 8.

[0047] Step 1: Obtain user comment original data, including user ID, comment ID, comment time, comment content, IP attribution of comment user, and merchant information; the comment content includes comment text and comment pictures.

[0048] In the data collection stage, the user comment original data can be obtained through the official data interface of each e-commerce platform, comment crawling program, etc., which can include comment ID, merchant information, user ID, comment time, comment content (comment text and / or comment pictures), stay duration, number of likes, number of collections, user mobile phone model, user IP, etc. The above information and data are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant provisions.

[0049] Step 2: Calculate the time decay factor according to the comment time.

[0050] The time decay factor defined in the present application is used to measure the degree of decay of the weight of the comment or data over time, and the value range is (0, 1]. The smaller it is, the lower the timeliness of the comment is, and the smaller the reference value for current decision-making is. The calculation formula of the time decay factor is as follows:

[0051] (1);

[0052] Wherein, This is the comment time, which is the original time the comment was posted, used to calculate the difference between the original time and the current time. This is the current time, i.e., the timestamp at the moment of calculation. This refers to the industry decay rate, used to reflect the rate at which the effectiveness of information declines across different industries, expressed in months or days. For example, for the catering industry... = 0.1 / month, information becomes outdated quickly, such as food reviews changing rapidly over time; for the education industry, = 0.03 / month, information decays slowly, such as teacher evaluations being valid for a long time.

[0053] The following example illustrates this. The calculation method. A certain catering platform ( = 0.1 / month) One post published on January 1, 2025 needs to be calculated. Restaurant reviews on May 27, 2025 ( Timeliness weight Then the time difference = 5 months (approximate value; precise calculation requires conversion to days). Substituting into formula (1) yields... That is, after 5 months, its reference value has decreased to about 60% of its initial value.

[0054] Step 3: Perform text processing and sentiment analysis on the comments to calculate the sentiment scores for positive and negative reviews.

[0055] Step 3 specifically includes steps 3.1 to 3.3.

[0056] Step 3.1: Perform text segmentation on the comment text and remove irrelevant symbols and stop words to obtain a word list after text processing.

[0057] First, use the jieba word segmenter to segment the comment text into individual words or phrases. This can be done using the jieba.cut() function in Python. For example, if the comment text is “The scenery is absolutely beautiful, but the hotel service is perfunctory and the food quality is ridiculously bad!”, the segmented output would be: ['scenery', 'absolutely beautiful', ',', 'hotel', 'service', 'perfunctory', ',', 'food', 'quality', 'bad', 'good', 'ridiculous', '!'].

[0058] Then clean the segmented text, including removing irrelevant symbols and stop words. Removing irrelevant symbols means deleting punctuation marks (such as commas, periods, exclamation marks), numbers, special characters, etc., which do not carry emotional information. In the above example, the symbols ",", "!" are removed. Stop words are common high-frequency function words (such as auxiliary words, prepositions, conjunctions), which do not contribute to emotional content. The stop word list usually uses a predefined list, such as the Harbin Institute of Technology stop word list or a custom list. Common Chinese stop words include "de", "le", "dan", "de", "he", etc. In the above example, in the stage of removing stop words, "de" (a particle indicating degree) is removed. "Dan" was not included in the original word segmentation, but similar words such as "de" are filtered. After removing irrelevant symbols and stop words, we get: ['scenery', 'extremely beautiful', 'hotel','service','perfunctory', 'food and beverage', 'quality', 'poor', 'ridiculous'].

[0059] Step 3.2: Screen out the comment keywords and key emotional words in the word list based on word nature or dictionary, and determine the polarity and emotional intensity score of each key emotional word; the polarity of key emotional words includes positive, negative and neutral.

[0060] In text sentiment analysis, "emotional words" refer to words in the text that directly or indirectly express emotional tendencies, emotional attitudes or evaluations. These words are usually content words (such as adjectives, verbs, nouns), which can reveal the author's emotion towards a certain topic, such as positive (positive), negative (negative) or neutral. The division of the polarity and intensity of emotional words is as Figure 2 shown. Emotional words mainly include:

[0061] - Direct emotional words: Words that clearly express emotions, such as adjectives ("extremely beautiful" indicates positive, "poor" indicates negative) or verbs ("like" indicates positive, "hate" indicates negative);

[0062] - Indirect emotional words: Nouns or phrases closely related to emotions, which may not directly express emotions themselves, but serve as the object or theme of emotions in the context (such as "service" or "quality"), helping to locate the specific target of emotions;

[0063] - Function: These words combined can reflect the overall emotional tendency of the text; for example, "perfunctory" in "service perfunctory" is a direct manifestation of negative emotion, and "service" specifies the emotional object.

[0064] Emotional words are defined by an emotional dictionary, and the jieba word segmentation package is used in conjunction with the built-in emotional dictionary to automatically extract emotional words and corresponding emotional intensity scores. In this application, direct emotional words are defined as "key emotional words", and indirect emotional words are defined as "comment keywords" for use.

[0065] From the cleaned word list, filter out the comment keywords and key sentiment words, which can be achieved in the following ways:

[0066] Based on Part-of-Speech (POS): Prioritize retaining adjectives (expressing sentiment), nouns (sentiment objects), and verbs (sentiment actions); for example, "exceedingly beautiful" (adjective), "perfunctory" (adjective), "bad" (adjective), "ridiculous" (adjective), are directly marked as key sentiment words; "scenery", "hotel", "service", "catering", "quality" (nouns) are retained as sentiment carriers (comment keywords);

[0067] Based on a sentiment dictionary (optional): Use a pre-trained Chinese sentiment dictionary (such as the HowNet sentiment dictionary or the BosonNLP dictionary) to match each word. If the word is in the dictionary and has a sentiment polarity (positive / negative), it is regarded as a key sentiment word; for example, "exceedingly beautiful" may be marked as positive in the dictionary, and "bad" as negative;

[0068] Based on simple rules: In the basic implementation, retain all content words (non-stop words) because they are often sentiment-related in the context; for example, "scenery", "hotel", "service", "catering", "quality" are retained as comment keywords related to key sentiment words.

[0069] Finally, output the list of key sentiment words and the corresponding list of comment keywords. For the key sentiment words, their sentiment intensity scores can be obtained by looking up the table in the word segmentation results. In the example of this application, the output results of the list of key sentiment words and the list of comment keywords are as follows:

[0071] {'target':'scenery', 'emotion_word': 'exceedingly beautiful', 'polarity': 1,'strength':1.2},

[0072] {'target': 'hotel service', 'emotion_word': 'perfunctory', 'polarity': -1,'strength': 0.9},

[0073] {'target': 'catering quality', 'emotion_word': 'bad', 'polarity': -1,'strength': 1.5},

[0074] {'target': 'catering quality', 'emotion_word': 'ridiculous', 'polarity': -1,'strength': 1.0}​ ]

[0076] wherein 'emotion_word' is the key emotion word, 'target' is the corresponding review keyword; 'polarity' represents the polarity, positive is 1, negative is -1, and neutral is 0.'strength' is the corresponding emotional intensity score, the value range is 0-10, and the specific range is divided as shown in Figure 2 .

[0077] Step 3.3: The good review sentiment score and the bad review sentiment score are obtained by summarizing the polarity and emotional intensity score of each key emotion word.

[0078] The calculation method of the good review sentiment score and the bad review sentiment score is: according to the evaluation object (review keyword), the positive / negative emotional intensity score is summarized, that is, the good review emotional intensity score or the bad review emotional intensity score corresponding to each review keyword is obtained. For example, in the above example, there are 2 records of dining quality, which are bad (intensity 1.5) and ridiculous (intensity 1), and both are negative, so the bad review emotional intensity score of dining quality is 2.5. When the polarity of the key emotion word is positive, it represents a good review, which is represented by 1; when the polarity of the key emotion word is negative, it represents a bad review, which is represented by 0; the summary result is shown in Table 1.

[0079] Table 1 Good review emotional intensity score / bad review emotional intensity score summary result

[0080]

[0081] Further, the good review emotional intensity score / bad review emotional intensity score in the whole review is summarized, that is, the good review sentiment score and the bad review sentiment score of the review are calculated, which are used for subsequent credibility score calculation. For example, for the example in Table 1, for the user review "The scenery is beautiful, but the hotel service is perfunctory, and the dining quality is bad and ridiculous!", the final good review sentiment score is 1.2 points, and the bad review sentiment score is 3.4 points (the emotional intensity scores of 2 evaluation objects are 0.9+2.5=3.4 points).

[0082] Step 4: The image-text association of the review picture and the review text is verified, and the image-text score is calculated.

[0083] The present application uses a CLIP (Contrastive Language-Image Pre-training) model to extract the visual and text features of the review content. CLIP maps different modal data to the same semantic space through a double-encoder architecture, obtaining two vectors in the same semantic space, namely an image feature vector and a text feature vector. Step 4 specifically includes the following steps 4.1 to 4.5.

[0084] Step 4.1: Extract visual features from the comment images to obtain image feature vectors.

[0085] First, load the CLIP computing environment, then load the pre-trained CLIP model. Input the comment text and image into the pre-trained CLIP model to preprocess the data and images, such as analyzing and recognizing mixed Chinese and English comments, and identifying image content. CLIP's visual encoder (ViT or CNN) can map the entire comment image into a fixed-dimensional visual feature vector.

[0086] Step 4.2: Extract text features from the comment text to obtain text feature vectors.

[0087] Similarly, CLIP's text encoder (Transformer) can map comment text into text feature vectors of the same dimension as the visual feature vectors. Since the image feature vectors and text feature vectors reside in the same semantic space, cosine similarity can be directly calculated to determine the degree of image-text matching.

[0088] Step 4.3: Normalize the image feature vector and text feature vector, and then calculate the image-text similarity between each image feature vector and text feature vector after normalization.

[0089] Feature vector normalization is the process of converting the feature vectors output by the CLIP model into unit vectors, that is, making the magnitude (L2 norm) of each feature vector 1. Without normalization, when calculating cosine similarity using the dot product, the dot product result is affected by the vector length and cannot accurately reflect semantic similarity.

[0090] After normalization, the cosine similarity between each image feature vector and text feature vector is calculated as the image-text similarity. Example code for cosine similarity calculation is as follows:

[0091] Similarity = (text_features @ image_features.T).item( )

[0092] print(f"Image-text similarity score: {similarity:.4f}") # Example output: 0.2100

[0093] Here, `text_features` represents the text feature vector output by the text encoder, with dimension `d` (commonly 512 or 768); when L2 normalization has been performed, it can be considered a unit vector. `image_features` represents the image feature vector output by the visual encoder, with the same dimension as `text_features`. `.T` represents transpose, for example, changing (1, d) to (d, 1), allowing subsequent matrix multiplication to calculate the inner product. `@` represents matrix multiplication (matmul), which in PyTorch is `torch.matmul`; when both vectors are unit vectors, the result of `@` is the cosine similarity. `(...).item()` is a tensor method used to convert a tensor containing only one element into a native Python floating-point number for easier printing or subsequent calculations. `Similarity` is the image-text similarity score, ranging from approximately [-1, 1]. `print(...)` is a built-in Python function used for console output. `f"..."` represents string interpolation, embedding variable values ​​into text. `{similarity:.4f}` is a format specifier used to retain 4 decimal places for floating-point numbers. In other words, the code above uses matrix multiplication to calculate the cosine similarity between the text feature vector and the image feature vector, and prints the result as a floating-point number with 4 decimal places.

[0094] Step 4.4: When the image-text similarity is greater than the similarity threshold, determine that the corresponding comment image is a comment image that matches the text.

[0095] This application sets the similarity threshold to 0.35, and counts the number of comment images with a similarity greater than 0.35 as the number of comment images with identical text and images. .

[0096] In the example review provided above, the user's text reads: "The scenery is absolutely stunning, but the hotel service is perfunctory and the food is terribly bad!" The corresponding review image is as follows: Figure 3 As shown, there are three comment images, each corresponding to... Figure 3(a), (b), (c). The pre-trained CLIP model is used to identify the content of the three review pictures, and the extracted picture content is: "five-star hotel pool" (85%), "Gulang Island Daylight Rock" (92%), and "seafood platter" (78%). For the image feature vectors corresponding to the above picture content, the text-image similarity is calculated respectively, where the "five-star hotel pool" has a text-image similarity score of 0.24 (<0.35), marked as text-image semantic fragmentation, and returns 0; "Gulang Island Daylight Rock" has a text-image similarity score of 0.13 (<0.35), marked as text-image semantic fragmentation, and returns 0; "seafood platter" has a text-image similarity score of 0.41 (>0.35), marked as text-image semantic consistency, and returns 1. The number of review pictures with text-image consistency =1, and the total number of review pictures =3.

[0097] Step 4.5: Calculate the text-image score using formula (2) , which is used for subsequent credibility score calculation:

[0098] (2);

[0099] where is the number of review pictures with text-image consistency; is the total number of review pictures. In the above example, the text-image score .

[0100] It should be noted that there are two special cases in step 4, i.e. only review pictures without review text, or only review text without review pictures, in which case there is no need to further check the text-image association, and the text-image score is directly output as =0.

[0101] Step 5: According to the space association verification of the review pictures and the IP attribution of the review user, the geographical score is calculated.

[0102] When extracting the image features of the comment picture, the EXIF (Exchangeable Image File Format) information is checked synchronously to prevent tampering. EXIF is specifically set for digital camera photos and can record the attribute information and shooting data of digital photos, such as shooting time, GPS position (latitude and longitude), device model, etc. Specifically, the GPS coordinates (degree-minute-second format) contained in the result of EXIF analysis are converted into decimal numerical values. Then the decimal GPS position is compared with the IP attribution of the comment user, and the deviation value is calculated, that is, the distance between the GPS position of the comment picture and the center of the IP attribution of the comment user is calculated , in meters. On this basis, the geographical score can be calculated using formula (3) :

[0103] (3);

[0104] wherein represents taking the maximum value; represents the zeroing distance threshold. In the above formula, the zeroing distance threshold is the critical distance (such as 10000 meters) at which the geographical score is reduced to 0; the score is deducted by 100 / zeroing distance threshold for every 1 meter of distance increase. In an example, assuming = 5000 meters, = 10000 meters, then = 50.

[0105] Step 6: Determine the industry coefficient according to the merchant information.

[0106] The industry coefficient is determined according to the industry type to which the merchant belongs, such as a weight of 0.25 for the catering industry and 0.15 for the education industry. The setting logic of the industry coefficient is: the catering industry strengthens the space-time authenticity verification, resists single brushing, and improves decision-making efficiency; the education industry weakens the geographical restriction, highlights the content quality and long-term credit value. The common industry types and corresponding industry coefficients are summarized in Table 2 as follows.

[0107] Table 2 Common industry types and corresponding industry coefficients

[0108]

[0109] Step 7: Calculate the credibility score according to the time decay factor, positive review sentiment score, negative review sentiment score, image-text score, geographical score, and industry coefficient.

[0110] The credibility score calculation formula is as follows:

[0111] (4);

[0112] wherein, is the time decay factor determined in step 2. is the geographic score determined in step 5. is the industry coefficient determined in step 6. represents the user history credibility, which is used for user behavior correction, and is determined according to the average credibility of the same user's comments in the past three months. High-credibility users = 1.1, low-credibility users = 0.8, and new users by default = 1. is the picture-text score calculated in step 4. represents the picture authenticity coefficient, which is determined according to the EXIF information verified in step 5. If the comment picture has EXIF information, it is 1, otherwise there is a risk of stealing network pictures, and the weight is 0.5; in addition, if there is no comment picture, the weight is 0. is the positive review sentiment score determined in step 3; represents the positive review weight proportion; is the negative review sentiment score determined in step 3; represents the negative review weight proportion. Generally, the positive review weight proportion and the negative review weight proportion are both 0.5. is the calculated credibility score.

[0113] When calculating the credibility score, it is necessary to pay attention to the normalization correction of each part of the score. For example, for the first part of the score , the second part of the score , the third part of the score and the fourth part of the score , the respective scores are denoted as , the scores need to be Min-Max normalized to normalize the scores of each part to between 0 and 1. The normalization correction formula is as follows:

[0114] (5);

[0115] For each part of the score , and represent the maximum and minimum values respectively, is the normalized score. The sum of the scores of each part is multiplied by 100, and the final credibility score is in the range of 0-100. For example, in a certain example, .

[0116] Step 8: Identify valid and invalid comments according to the credibility score.

[0117] When the credibility score is lower than the score lower threshold (usually 60 points), the corresponding user comment is determined to be an invalid comment, and the invalid comment is folded or hidden; otherwise, the corresponding user comment is determined to be a valid comment, and the valid comment is displayed from high to low according to the credibility score. The recognition accuracy of invalid comments is improved to more than 98% by the method of the present application, and the traditional method is about 85%.

[0118] In real life, invalid comments (such as "good review return" induction, water army brushing) often drown out real user feedback. By using the method of the present application, low credibility comments (such as <60 points) are identified as invalid comments, automatically folded or marked, reducing interference, and can achieve comment noise reduction. For example, an e-commerce platform Step down) combined with a low image-text score (no image / template text) to calculate a credibility score lower than 60, it is determined to be a water army and hidden.

[0119] When an e-commerce platform needs to efficiently obtain valuable information (such as product defects, use experience), it can also filter high-quality comments through the method of the present application, and set a score upper threshold (usually 80 points) to mark high credibility comments with a score higher than 80 as "high-quality comments" for priority display.

[0120] Of course, the calculation of each score in the method of the present application can also be used individually or in any combination according to the actual application scenario. For example, for comments with high geographic score (real user) + high image-text score (with image / video), push to the top of the detail page for display. For comments with high negative review sentiment score (such as "express violence sorting" complaints), trigger customer service to actively contact the user. For low geographic score (IP frequently switched) + high sentiment score but repeated text user comments, determine as brushing. At the group level, clustering analysis of comments with similar credibility scores can also identify "water army groups", such as 10 comments are all abnormal. For example, by optimizing the platform recommendation weight through the credibility score, high-quality comments containing real videos and detailed use experience are preferentially displayed; or quickly select high-quality creator features (such as image-text originality, excellent cases). By detecting inconsistent image-text expressions, malicious or false comments can be identified; such as the comment image is a landscape, but the comment content describes food spoilage; the comment user IP belongs to overseas, but frequently comments on specified domestic restaurants. Detect and count specified user comments, extract positive and negative keywords to form user purchase preferences.

[0121] ​The method can realize multi-modal data fusion noise reduction, based on text consistency verification, combined with comment image and text semantics, through cross-modal comparison to remove false content (such as "mobile phone shell" comment image for shoes), and the accuracy rate reaches 98%. Further, based on behavior feature weighting, integrating user comment trajectory (user IP, merchant information, comment time, comment content) and comment sentiment polarity, dynamically adjusting the weight and parameter value of each item in the credibility score calculation formula, the accuracy of effective / invalid comment identification is greatly improved. The method only needs to obtain the original data of user comments through the official data interface of each platform, and can quickly calculate the credibility score to determine whether the merchant has a fake comment behavior, which can more effectively supervise the merchants in the platform, reflect the real product situation, and has high application value.

[0122] In an exemplary embodiment, the present application also provides a computer device which can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface and a communication interface. Wherein the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Wherein the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to realize the e-commerce comment noise reduction method.

[0123] In an exemplary embodiment, the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by the processor to realize the e-commerce comment noise reduction method.

[0124] In an exemplary embodiment, the present application also provides a computer program product comprising a computer program, wherein the computer program is executed by the processor to realize the e-commerce comment noise reduction method.

[0125] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by computer program instruction related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory or other medium in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read only memory (Read Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0126] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0127] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0128] The principles and implementation modes of the present application are described by specific examples in this paper. The above description of the embodiments is only used to help understand the method and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for de-noising e-commerce reviews, characterized in that, The method comprises the following steps: Obtain user comment raw data, including user ID, comment ID, comment time, comment content, IP belonging place of the comment user, and merchant information; The comment content includes comment text and comment pictures; Calculate a time decay factor according to the comment time; The method of calculating the time decay factor according to the comment time specifically comprises: Using formula Calculate the time decay factor ;in Comment time; The current time; Industry decline rate; Perform text processing and sentiment analysis on the comment text to calculate a positive sentiment score and a negative sentiment score; Perform image-text correlation verification on the comment pictures and the comment text to calculate an image-text score; Perform spatial correlation verification on the comment pictures and the IP belonging place of the comment user to calculate a geographic score; Determine an industry coefficient according to the merchant information; Calculate a credibility score according to the time decay factor, the positive sentiment score, the negative sentiment score, the image-text score, the geographic score, and the industry coefficient; Identify valid comments and invalid comments according to the credibility score.

2. The method of claim 1, wherein, The method of performing text processing and sentiment analysis on the comment text to calculate a positive sentiment score and a negative sentiment score specifically comprises: Perform text segmentation on the comment text and remove irrelevant symbols and stop words to obtain a word list after text processing; Screen out comment keywords and key sentiment words in the word list based on word classes or dictionaries, and determine the polarity and sentiment intensity score of each key sentiment word; the polarity of the key sentiment word includes positive, negative, and neutral; Summarize the positive sentiment score and the negative sentiment score according to the polarity and sentiment intensity score of each key sentiment word.

3. The method of claim 2, wherein, The method of performing image-text correlation verification on the comment pictures and the comment text to calculate an image-text score specifically comprises: Extract visual features from each comment picture to extract corresponding image feature vectors; Extract text features from the comment text to extract text feature vectors; Perform normalization processing on the image feature vectors and the text feature vectors, and calculate the image-text similarity between each image feature vector and text feature vector after normalization; When the image-text similarity is greater than a similarity threshold, determine that the corresponding comment picture is an image-text consistent comment picture. The graphic-text score is calculated using the formula ; wherein is the number of review images with consistent graphic and text; is the total number of review images.​ 4. The method of claim 3, wherein, The method of performing spatial correlation verification on the comment pictures and the IP belonging place of the comment user to calculate a geographic score specifically comprises: extracting a GPS location of the review picture and calculating a distance between the GPS location of the review picture and a center of an IP attribution of the review user ; The geographic score is calculated using the formula The geographic score is calculated using the formula ; wherein denotes taking the maximum value; denotes a zeroing distance threshold.

5. The method of claim 4, wherein, The method of calculating a credibility score according to the time decay factor, the positive sentiment score, the negative sentiment score, the image-text score, the geographic score, and the industry coefficient specifically comprises: The formula is as follows: The credibility score is calculated ; wherein represents an industry coefficient; represents a user historical credibility; represents a picture authenticity coefficient; represents a good review sentiment score; represents a good review weight proportion; represents a bad review sentiment score; represents a bad review weight proportion.

6. The method of claim 5, wherein, The method of identifying valid comments and invalid comments according to the credibility score specifically comprises: When the credibility score is lower than a score lower limit threshold, determine that the corresponding user comment is an invalid comment, and fold or hide the invalid comment; otherwise, determine that the corresponding user comment is a valid comment, and display the valid comment according to the credibility score from high to low.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the e-commerce comment noise reduction method of any one of claims 1 to 6.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the e-commerce comment noise reduction method of any one of claims 1 to 6.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the e-commerce comment noise reduction method of any one of claims 1 to 6.

Images