Data filtering method, system, equipment and product for automobile user comments

Abstract

The invention provides a data filtering method, system, device and product for automobile user comments, relates to the technical field of comment data processing, and realizes efficient identification and continuous optimization elimination of unreal user comments by constructing a dynamic filtering mechanism oriented to the automobile field. The method comprises the following steps: firstly, carrying out preliminary screening on multi-source heterogeneous comments based on a preset rule base, then carrying out semantic segmentation and high-frequency word statistics in combination with a field customized stop word list, automatically mining potential non-real user identification keywords by utilizing an inter-class distinction degree index, and dynamically updating the rule base; and through multi-round iterative filtering, it is ensured that a newly appearing false comment mode can be captured in time and brought into an identification range until the system converges. According to the technical effects, the purity and representativeness of a real user comment data set are remarkably improved, and a high-credibility data basis is provided for subsequent user insight, product improvement and market decision.

Description

Technical Field

[0001] This invention relates to the field of comment data processing technology, and in particular to a data filtering method, system, device and product for car user comments. Background Technology

[0002] In today's digital age, automotive user reviews are widely distributed across heterogeneous data sources such as professional automotive websites, social media, and e-commerce platforms, holding significant value for consumer car-buying decisions, automakers' product improvements, and market trend analysis. However, existing technologies face significant challenges in integrating multi-source review data: On the one hand, a large number of reviews come from anonymous or unverifiable users, including car salespeople, competitor attackers, and content template publishers—non-genuine users whose reviews often contain promotional, derogatory, or highly similar content; on the other hand, existing filtering methods generally rely on static keyword rules or general natural language processing tools, lacking a deep understanding and dynamic update mechanism for the semantic features of the automotive field, making it difficult to effectively identify fake reviews and drowning out genuine user feedback. Furthermore, traditional stop word lists are not adapted to automotive terminology, easily leading to the accidental deletion of key information, and lack a comprehensive filtering strategy that combines user behavior, knowledge graph consistency verification, and multi-dimensional credibility assessment, resulting in insufficient accuracy of the final dataset and affecting the reliability of downstream analysis and the scientific nature of decision-making. Summary of the Invention

[0003] This invention aims to solve the technical problems existing in the above-mentioned related technologies, and proposes a data filtering method, system, device and product for car user reviews. It can efficiently extract high-fidelity real user feedback data by dynamically identifying and eliminating non-genuine user reviews, combined with domain-customized semantic segmentation, knowledge graph verification and multi-dimensional credibility assessment.

[0004] The solution to the technical problem of this invention is: This invention provides a data filtering method for car user reviews, comprising the following steps:

[0005] Acquire automotive user review data from heterogeneous data sources, the review data including user identifiers, text review content, and additional tags; The comment data is initially filtered based on a pre-set non-real user identification rule base. The non-real user identification rule base stores explicit identity identifier keywords, which include words associated with car sales and service, commercial promotion content production, or competitor denigrating behavior. The initially filtered comment data is semantically segmented using a stop word list customized for the automotive industry to generate candidate word sequence, and the frequency of occurrence of each candidate word is counted. Based on the frequency distribution and inter-class discrimination index of the candidate word groups, new explicit identity identification keywords are identified, and the new keywords are dynamically updated to the non-real user identification rule base. Based on the updated non-real user identification rule base, the preliminary filtering, semantic segmentation and keyword identification steps are executed iteratively until no new explicit identity identification keywords are identified in a single complete iteration. The output is the final filtered and retained comment data as a dataset of real user comments.

[0006] Furthermore, the inter-class discrimination index is at least one of TF-IDF value, information gain, or chi-square statistic, and a candidate word group is only identified as a new explicit identity keyword when its inter-class discrimination index in the suspicious comment subset exceeds a preset threshold.

[0007] Furthermore, after obtaining car user review data, the method also includes: obtaining users' historical review behavior data; if the same user publishes reviews with similar content structures on more than three different car brands within a preset time window, the user is marked as a suspicious user and their review data is preferentially excluded in the preliminary filtering.

[0008] Furthermore, the semantic segmentation adopts a word segmentation strategy that combines automotive domain dictionary matching with local context windows to maintain the integrity of the binary semantic unit of "vehicle technical features - user evaluation opinions".

[0009] Furthermore, based on a pre-built automotive domain knowledge graph, the semantic consistency of the binary semantic unit "vehicle technical features - user evaluation opinions" is further verified. If the two have no reasonable connection path in the knowledge graph or there is a factual conflict, the credibility weight of the corresponding comment is reduced or it is removed.

[0010] Furthermore, before outputting the real user review dataset, the method further includes: calculating a comprehensive credibility score for each retained review, the credibility score being determined based on a weighted average of at least two of the following: (1) Whether the user has completed vehicle binding or real-name authentication; (2) The number of specific vehicle parameters or usage scenario details included in the comment content; (3) Whether the length falls within the typical range of real users; (4) The logical consistency between the comment posting time and the vehicle delivery time; A review will only be included in the real user review dataset if its credibility score is higher than a preset threshold.

[0011] Furthermore, the stop word list customized for the automotive field excludes automotive technical feature terms, performance evaluation terms, and user experience descriptions, retaining only grammatical function words and punctuation marks that do not contribute to the semantics.

[0012] On the other hand, this application provides a data filtering system for car user reviews, including: The data acquisition unit is configured to acquire car user review data from heterogeneous data sources, wherein the review data includes user identifiers, text review content, and additional tags; The preliminary filtering unit is configured to perform preliminary filtering on the comment data based on a preset non-real user identification rule base. The non-real user identification rule base stores explicit identity identifier keywords, which include words associated with car sales and service, commercial promotion content production, or competitor denigrating behavior. The semantic segmentation unit is configured to perform semantic segmentation on the initially filtered comment data using a stop word list customized for the automotive field, generate a sequence of candidate word groups, and count the frequency of occurrence of each candidate word group. The rule update unit is configured to identify new explicit identity keywords based on the frequency distribution and inter-class distinguishability index of the candidate word groups, and dynamically update the new keywords to the non-real user identification rule base. The iterative control unit is configured to trigger the preliminary filtering unit, semantic segmentation unit, and rule update unit to perform cyclic processing until no new explicit identity keywords are identified during a single complete iteration. The data output unit is configured to output the final filtered and retained comment data as a dataset of real user comments.

[0013] On the other hand, this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned data filtering method for car user reviews.

[0014] On the other hand, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned data filtering method for car user reviews.

[0015] The beneficial effects of this invention are as follows: This application provides a data filtering method for automotive user reviews. This method achieves efficient identification and continuous optimization and elimination of non-genuine user reviews by constructing a dynamic filtering mechanism oriented towards the automotive field. The method first performs preliminary screening of multi-source heterogeneous reviews based on a preset rule base, then combines a domain-customized stop word list for semantic segmentation and high-frequency word statistics, and automatically mines potential non-genuine user identifier keywords using inter-class discrimination indicators, dynamically updating the rule base. Through multiple rounds of iterative filtering, it ensures that newly emerging fake review patterns can be captured and included in the identification range in a timely manner until the system converges. This technology significantly improves the purity and representativeness of the real user review dataset, effectively solving the problems of easy omissions and weak generalization ability in traditional static rule filtering, providing a highly reliable data foundation for subsequent user insights, product improvement, and market decisions. This application also provides corresponding systems, devices, and products. The beneficial effects of the systems, devices, and products are the same as those of the above method, and will not be elaborated here.

[0016] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0017] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0018] Figure 1 This is a flowchart of the data filtering method for car user reviews provided in this application; Figure 2 This is a structural diagram of the data filtering system for car user reviews provided in this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0021] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] In an era of rapid digitalization and intelligentization, user-generated content (UGC) has become a key source of information influencing consumer decisions, product iterations, and marketing strategies. This is particularly true in the automotive industry, where consumers rely heavily on real user experiences, performance evaluations, and troubleshooting feedback before purchasing a vehicle; automakers, on the other hand, analyze massive amounts of user reviews to identify product defects, optimize designs, adjust configurations, and develop precise marketing strategies. Therefore, obtaining high-quality, reliable, and authentic user review data has become a crucial link between user needs and corporate responses.

[0024] However, current automotive reviews are widely distributed across various heterogeneous platforms—including specialized automotive websites, general social media, e-commerce platforms, and forums. These platforms exhibit inconsistent data structures, varying quality, strong anonymity, and a lack of unified content moderation mechanisms. In this environment, a large number of inauthentic reviews are mixed in, severely interfering with the authenticity and usability of the data.

[0025] More complexly, some reviews do not originate from actual car owners or users, but are posted by stakeholders whose purpose may be to promote specific car models, suppress competitors, create false reputation, or meet KPI targets. These reviews often exhibit clear identity tags, templated language structures, excessively emotional expressions, or repetitive content across brands, but traditional data processing methods struggle to effectively identify and filter them. Therefore, there is an urgent need for a car user review filtering technology that can integrate multi-source data, dynamically identify non-genuine users, retain genuine feedback, and possess continuous learning capabilities to support subsequent high-value data analysis and commercial applications.

[0026] Current data processing for automotive user reviews primarily relies on static keyword filtering, general natural language processing tools (such as standard word segmenters and stop word lists), anomaly detection based on user behavior (such as high-frequency posting or cross-brand review identification), machine learning classification models (such as SVM and BERT for classifying genuine and fake reviews), and real-name authentication or vehicle binding verification mechanisms used by some platforms. While these methods can be used to some extent for preliminary screening of spam or suspicious accounts, they are mostly used in isolation, lack systematic integration, and generally are not optimized for the specialized semantic features of the automotive field.

[0027] Existing technologies suffer from several limitations: First, static keyword rule bases cannot adapt to new types of deceptive rhetoric, have weak generalization capabilities, and are slow to update, easily offending genuine users or missing hidden marketing content. Second, general NLP tools are not adapted to automotive terminology, often incorrectly removing key evaluation terms, and their word segmentation strategies ignore core semantic structures such as "technical features—user opinions." Third, the lack of semantic consistency verification based on automotive knowledge graphs makes it impossible to identify comments with factual errors or logical contradictions. Fourth, most solutions only analyze text content, ignoring multi-dimensional credibility signals such as user identity, behavioral patterns, and temporal logic. Fifth, the filtering process lacks a closed-loop iterative mechanism, failing to automatically extract and update identification rules from newly discovered suspicious patterns. Finally, there is a lack of effective ability to identify biased comments from non-genuine users, leading to data bias. These shortcomings collectively make it difficult for existing methods to guarantee the authenticity and analytical value of automotive review data.

[0028] To address the aforementioned issues, this application proposes a data filtering method that integrates domain knowledge and adaptive learning capabilities. Its main technical features are as follows: First, it constructs a rule base for identifying non-real users in the automotive field, focusing on explicit identity keywords such as sales, promotion, and competitor attacks for initial filtering. Second, it employs a stop word list specifically tailored for automotive reviews and a semantic segmentation strategy combined with a domain dictionary to retain key semantic units such as "vehicle technical features—user evaluation opinions." Third, it automatically mines potential non-real user tags and dynamically updates the rule base by statistically analyzing the frequency distribution of candidate word groups and inter-class discrimination indicators (such as TF-IDF and chi-square test). Furthermore, it introduces an iterative closed-loop mechanism to continuously optimize the filtering effect until convergence. Simultaneously, it integrates user behavior analysis, knowledge graph consistency verification, and multi-dimensional credibility scoring (such as real-name authentication, content details, and temporal logic) to achieve in-depth identification and hierarchical screening of review data. This solution combines professionalism, intelligence, and scalability, significantly improving the purity and usability of real user review data.

[0029] First, the data filtering method for car user reviews provided in this application will be described in detail below with reference to the accompanying drawings.

[0030] Reference Figure 1The implementation process of the data filtering method for car user reviews provided in this application embodiment includes, but is not limited to, the following steps.

[0031] Step S110: Obtain car user review data from heterogeneous data sources.

[0032] The comment data includes user identifiers, text comment content, and additional tags.

[0033] In step S110, raw data from car user reviews is systematically collected from multiple data sources of different types (i.e., "heterogeneous data sources"), providing basic input for subsequent filtering processing. These data sources may include professional automotive websites, social media platforms, e-commerce platforms, etc., covering a wide range of user groups and review scenarios. The acquired review data is structured to contain three core elements: user identifiers (used to track the identity information of the review publisher), text review content (the user's actual evaluation, experience, or opinion on the vehicle), and additional tags (such as user classification, review topic, sentiment tendency, and other auxiliary information). This step ensures the comprehensiveness and diversity of the data, which is a prerequisite for building a high-quality dataset of real user reviews.

[0034] Step S120: Perform preliminary filtering of comment data based on a preset non-real user identification rule base.

[0035] The non-genuine user identification rule base stores explicit identity-identifying keywords. These keywords include terms associated with car sales and services, commercial promotional content production, or competitor disparaging behavior.

[0036] In step S120, a pre-built rule base is used to perform the first round of filtering on the original comment data to quickly remove comments that are clearly not from real users. The core of the rule base is "explicit identity identifier keywords." These keywords are clearly targeted and directly related to the identity or behavioral characteristics of non-real users. For example, words related to car sales services (such as "4S store," "sales consultant"), commercial promotion content production (such as "car review," "review blogger," "cooperative promotion"), or competitor denigration (such as "XX brand is garbage," "never buy YY," and other offensive language). As long as the user identifier, text content, or additional tags in the comment contain such keywords, it is judged as suspicious and filtered. This step achieves efficient interception of high-confidence non-real comments and significantly reduces data noise in subsequent processing.

[0037] Step S130: The pre-filtered comment data is semantically segmented using a stop word list customized for the automotive field to generate candidate word sequence and count the frequency of each candidate word.

[0038] In step S130, the preliminarily cleaned comment text undergoes refined language processing. The key lies in using a "stop word list customized for the automotive industry"—this stop word list specifically excludes terms with substantial semantic value in automotive reviews (such as technical parameters, performance descriptions, and user experience vocabulary), retaining only meaningless grammatical function words (such as particles, prepositions, and conjunctions) and punctuation. Based on this, the text is semantically segmented to form meaningful "candidate word group sequences." Subsequently, the system performs frequency statistics on all candidate word groups, calculating the number of times each word group appears in the entire dataset. This process not only preserves key semantic units in the comments but also provides a quantitative basis for subsequent identification of potential non-genuine user tags.

[0039] Step S140: Based on the frequency distribution of candidate word groups and the inter-class distinguishability index, identify new explicit identity identifier keywords and dynamically update the non-real user identification rule base with the new keywords.

[0040] In step S140, novel non-authentic user identifier terms not yet covered by the initial rule base are mined from high-frequency candidate word groups. Specifically, the system not only examines the frequency of word group occurrences but also combines "inter-class discrimination indicators" (such as the concentration of a word group in a subset of suspicious comments) to determine whether it possesses identity identifier attributes. If a word group appears frequently in a specific comment group and has significant distinguishability, it is identified as a "new explicit identity identifier keyword." Once confirmed, the word is automatically added to the non-authentic user identification rule base, realizing the dynamic expansion and self-evolution of the rule base. This step enables the system to autonomously learn new noise patterns from data, overcoming the limitations of manual rule maintenance.

[0041] Step S150: Based on the updated non-real user identification rule base, iteratively execute the preliminary filtering, semantic segmentation and keyword identification steps until no new explicit identity identifier keywords are identified in a single complete iteration.

[0042] In step S150, a closed-loop iteration mechanism is introduced to ensure the sufficiency and convergence of the filtering process. After each rule base update, the system re-executes the complete process from initial filtering (S120) to keyword recognition (S140), using the enhanced rule base to screen the original or intermediate data again, thereby capturing previously missed non-genuine comments. This process is repeated until no new explicit identity keywords are found in a complete iteration. This design ensures continuous optimization of the rule base and thorough data cleaning, avoiding missed judgments caused by one-time processing, and making the final result stable and complete.

[0043] Step S160: Output the final filtered and retained comment data as a dataset of real user comments.

[0044] In step S160, the remaining comment data after multiple rounds of cleaning and verification is officially output, forming the final "real user comment dataset." This data has been effectively stripped of non-genuine components such as sales, promotional, and offensive comments, representing genuine feedback more likely from actual car owners or users. This dataset can be directly used for high-value applications such as user insights, product improvement, market analysis, or consumer decision support; its purity and representativeness are the core outputs of this method.

[0045] In some embodiments of this application, in step S140, the inter-class discrimination index is at least one of TF-IDF value, information gain or chi-square statistic, and the candidate word group is only determined as a new explicit identity keyword when the inter-class discrimination index of the candidate word group in the suspicious comment subset exceeds a preset threshold.

[0046] Specifically, TF-IDF (Term Frequency-Inverse Document Frequency) measures the inverse relationship between the frequency of a phrase in a subset of suspicious comments and its prevalence in the entire comment corpus. A high TF-IDF value indicates that the phrase appears frequently in suspicious comments but is relatively rare in the overall context. Information gain reflects the information contribution of the phrase to distinguishing between "suspicious comments" and "non-suspicious comments," with a higher value indicating greater classification value. Chi-square statistic tests the independence between the phrase and the suspicious comment category; a significant deviation from independence indicates a strong association. By setting a preset threshold, the system only identifies candidate phrases with inter-class discrimination in the suspicious comment subset that exceed the threshold as new explicit identity keywords, effectively avoiding misclassification of high-frequency but identity-neutral generic words (such as "good" or "not bad") as non-real user tags. This mechanism ensures high confidence and strong relevance of newly identified keywords, improving the accuracy of rule base updates and the robustness of the filtering system.

[0047] In some embodiments of this application, after obtaining the car user review data in step S110, the method further includes: obtaining the user's historical review behavior data; if the same user publishes reviews with similar content structures on more than three different car brands within a preset time window, the user is marked as a suspicious user and their review data is preferentially excluded in the initial filtering.

[0048] Specifically, the system tracks the cross-brand review activity of the same user within a preset time window (e.g., 30 or 90 days) and analyzes the structural similarity of their review content. If a user is found to have published highly similar reviews for three or more different car brands within that time window (e.g., using the same sentence structure, templated expressions, repeated viewpoints, or similar wording), then that user is highly likely not a real car owner, but rather a marketer or automated script operator engaged in mass content production. This behavioral pattern differs significantly from the typical behavior of real users (usually focusing on their own purchased car model, with personalized details and real-world usage scenarios in their reviews). Therefore, the system marks such users as "suspicious users" and prioritizes excluding their review data in the subsequent preliminary filtering stage (step S120). This measure effectively enhances the ability to identify non-genuine reviews that are highly concealed, lack explicit keywords, but exhibit abnormal behavior, compensating for the shortcomings of relying solely on text keyword filtering and improving the coverage and accuracy of the filtering system from the source.

[0049] In some embodiments of this application, in step S130, semantic segmentation adopts a word segmentation strategy that combines automotive domain dictionary matching with local context windows to maintain the integrity of the binary semantic unit of "vehicle technical features - user evaluation opinions".

[0050] Specifically, car user reviews often contain tightly coupled binary semantic units of "vehicle technical features - user evaluation opinions," such as "transmission jerking," "poor seat support," and "good sound insulation." The first part refers to specific technical attributes or components (e.g., "transmission," "seat," "sound insulation"), while the second part expresses subjective feelings or performance judgments (e.g., "jerking," "poor," "good"). Using standard word segmenters not adapted to this domain may incorrectly segment such phrases into isolated words, disrupting their inherent semantic connections. To address this, this solution introduces a specially constructed automotive domain dictionary, prioritizing the identification and preservation of complete technical terms. Simultaneously, it incorporates a local context window mechanism to dynamically consider the collocation relationships of adjacent words during segmentation, thereby accurately identifying and maintaining the integrity of these binary semantic units. This strategy not only improves the accuracy of phrase extraction but also lays a high-quality linguistic foundation for subsequent high-frequency word statistics, keyword mining, and semantic analysis.

[0051] In some embodiments of this application, the semantic consistency of the binary semantic unit “vehicle technical features - user evaluation opinions” is further verified based on a pre-built automotive domain knowledge graph. If the two have no reasonable connection path in the knowledge graph or there is a factual conflict, the credibility weight of the corresponding comment is reduced or it is removed.

[0052] Specifically, the system verifies the authenticity and rationality of comments from the perspective of semantic consistency and knowledge logic, preventing data distortion caused by factual errors or conflicts of common sense. The automotive knowledge graph systematically integrates structured knowledge such as vehicle parameters, technical principles, component functions, common fault modes, and their reasonable manifestations, representing the logical connections between different technical features and reasonable user feedback. For example, "electric vehicles have no gearbox" is a basic fact. If a comment claims "a certain pure electric vehicle's gearbox shifts smoothly," then there is no reasonable connection path between "gearbox" and "pure electric vehicle" in the knowledge graph, potentially constituting a factual conflict. By mapping extracted binary semantic units to the knowledge graph, the system can automatically detect such contradictions: if there is a lack of a reasonable semantic path between technical features and evaluation opinions, or if it clearly violates known engineering common sense, then the comment is deemed to have credibility issues. In this case, the system will correspondingly reduce the credibility weight of the comment, or directly remove it in strict mode. This mechanism significantly enhances the filtering system's ability to identify comments that "seem true but are absurd," ensuring that the final retained data is not only reliable in origin but also reasonable in content, thereby improving the professionalism and analytical value of the overall dataset.

[0053] In some embodiments of this application, before outputting the real user review dataset in step S160, the method further includes: calculating a comprehensive credibility score for each retained review, wherein the credibility score is determined based on a weighted average of at least two of the following: (1) Whether the user has completed vehicle binding or real-name authentication. This indicator directly reflects the authenticity of the user's identity. Users who have completed vehicle verification or real-name authentication on the official platform are more likely to be real car owners; (2) The more detailed the comments are, the more they reflect the real user experience, rather than generalized template content. (3) Consider whether the length is within the typical range of real users. If it is too short (such as just "Great car!"), it may lack substantial information. If it is too long and has a neat structure, it may be marketing copy. The system sets a reasonable length range based on the statistical distribution of historical real comments. (4) The logical consistency between the review posting time and the vehicle delivery time. For example, if the user picks up the vehicle in June 2024, then a detailed user review posted in May 2024 is obviously illogical, and such time conflicts will significantly reduce credibility.

[0054] The comment is included in the real user comment dataset only when the credibility score is higher than a preset threshold. This mechanism effectively filters out "marginal comments" that do not trigger keyword rules but have empty content, suspicious logic, or lack traces of real use, significantly improving the overall quality, authenticity of the dataset, and the reliability of subsequent analysis.

[0055] In some embodiments of the present application, the stop word list customized for the automotive field excludes automotive technical feature terms, performance evaluation vocabulary, and user experience description words, and only retains grammatical function words and punctuation marks that have no substantial semantic contribution.

[0056] Specifically, in the natural language processing process, the stop word list is usually used to filter out grammatical function words with high frequency but weak semantic contribution, such as "de", "le", "zai", "danshi", etc., to reduce the computational complexity and focus on key content. However, the general stop word list does not consider the semantic characteristics of specific fields. If directly applied to automotive review data, it is very likely to delete a large number of professional words with high information density. For example, "fuel consumption", "gearbox", "chassis tuning", "NVH performance", etc. belong to automotive technical feature terms; "powerful", "soft brakes", "precise steering", etc. belong to performance evaluation vocabulary; and "comfortable seats", "enough rear space", "obvious high-speed wind noise", etc. are typical user experience description words. These words are the core carriers for real users to express the advantages and disadvantages of vehicles, usage pain points, and satisfaction, and are also the key basis for subsequent product defect identification, user demand mining, and market trend analysis.

[0057] If the above words are wrongly included in the stop words and excluded, it will not only lead to serious loss of semantic information, but also make downstream tasks such as high-frequency word statistics, keyword extraction, and opinion mining ineffective or deviated. Therefore, the present application actively excludes these high-value words, clearly retains them in the analysis corpus, and only filters out real虚词 and punctuation marks with no substantial semantic contribution, so as to generate a more accurate and complete candidate phrase sequence in the semantic segmentation (such as step S130) stage. This domain-adapted stop word strategy significantly improves the accuracy and professionalism of text processing, laying a solid language foundation for subsequent identification of non-real user labels, extraction of real user attention hotspots, and construction of a high-quality comment dataset, and is an indispensable key link to achieve the technical effects of the present invention.

[0058] Secondly, referring to Figure 2 , the data filtering system for automotive user comments provided by the embodiments of the present application includes: A data acquisition unit configured to acquire automotive user comment data from heterogeneous data sources, and the comment data includes user identifiers, text comment content, and additional tags.

[0059] The preliminary filtering unit is configured to perform preliminary filtering of comment data based on a preset non-real user identification rule base. The non-real user identification rule base stores explicit identity identifier keywords, which include words associated with car sales and services, commercial promotion content production, or competitor derogatory behavior.

[0060] The semantic segmentation unit is configured to perform semantic segmentation on the initially filtered comment data using a stop word list customized for the automotive field, generate a sequence of candidate word groups, and count the frequency of occurrence of each candidate word group.

[0061] The rule update unit is configured to identify new explicit identity keywords based on the frequency distribution of candidate word groups and inter-class distinguishability indicators, and dynamically update the non-real user identification rule base with the new keywords.

[0062] The iterative control unit is configured to trigger the preliminary filtering unit, semantic segmentation unit, and rule update unit to perform cyclic processing until no new explicit identity keywords are identified during a single complete iteration.

[0063] The data output unit is configured to output the final filtered and retained comment data as a dataset of real user comments.

[0064] Furthermore, embodiments of this application provide an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the aforementioned data filtering method for car user reviews.

[0065] Furthermore, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned data filtering method for car user reviews.

[0066] In summary, the data filtering method, system, device, and product for car user reviews provided in this application have the following technical effects.

[0067] This application's embodiments significantly improve the purity and representativeness of real user review data by constructing an intelligent filtering mechanism that integrates domain knowledge and dynamic learning capabilities. The solution employs a stop word list customized for the automotive field and a semantic segmentation strategy combined with a domain dictionary, effectively preserving key semantic units such as "vehicle technical features—user evaluation opinions." After initial filtering based on a preset rule base, it further automatically identifies novel non-real user keywords through frequency statistics and inter-class discrimination indicators (such as TF-IDF and chi-square test), and dynamically updates the rule base. Simultaneously, a closed-loop iterative mechanism is introduced to continuously optimize the filtering effect until convergence, thereby overcoming the problems of weak generalization ability and lagging updates in traditional static rules.

[0068] Furthermore, this application enhances the ability to identify covertly fraudulent reviews: by analyzing users' historical behavior (such as posting structurally similar reviews for multiple brands within a preset time window), verifying semantic consistency using automotive domain knowledge graphs, and calculating multi-dimensional comprehensive credibility scores (including real-name authentication, content details, review length, and temporal logic) before output, it performs refined screening of marginal reviews. The final output dataset not only eliminates explicit marketing and offensive content but also filters out low-credibility reviews with logical contradictions, factual errors, or a lack of genuine usage traces, providing automakers with a highly accurate, reliable, and business-value foundation of real user feedback data for product improvement, market decisions, and consumer car purchase references.

[0069] It should be noted that in all specific embodiments of this application, all data processing activities related to user identity or personal characteristics, such as user information, user behavior data, historical data, and location information, will be conducted in accordance with the principles of legality, legitimacy, and necessity. All data collection, use, storage, and processing will be subject to compliance with applicable national and regional laws, regulations, and industry standards, and informed consent from users will be obtained in a clear and explicit manner before processing. For the processing of sensitive personal information, separate consent from users will be obtained through prominent means such as pop-up prompts and independent confirmation pages. If any processing conflicts with laws and regulations, the laws and regulations will prevail, and necessary data processing will only be carried out within the scope permitted by laws and regulations, ensuring that all data-based applications, analyses, and technical implementations are conducted within the scope permitted by laws and regulations.

[0070] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0071] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary skill. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0072] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0073] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable programs for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can retrieve and execute a program from or in conjunction with such a program execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with a program execution system, apparatus, or device.

[0074] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Additionally, computer-readable media can even be paper or other suitable media on which programs can be printed, for example, by optically scanning the paper or other media, then editing, interpreting, or, if necessary, processing it in a suitable manner to obtain the program electronically, and then storing it in computer memory.

[0075] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0076] In the foregoing description of this specification, the reference to terms such as "one embodiment / implementation," "another embodiment / implementation," or "certain embodiments / implementations," etc., indicates that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in an embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0077] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0078] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A data filtering method of automobile user reviews, characterized by, The method comprises the following steps: Obtaining automobile user comment data from heterogeneous data sources, the comment data comprising user identification, text comment content and additional labels; Preliminary filtering of the comment data based on a preset non-real user identification rule library, the non-real user identification rule library storing explicit identity identification keywords, the explicit identity identification keywords comprising words associated with automobile sales services, commercial promotion content production or competitor derogatory behavior; Semantic segmentation of the preliminarily filtered comment data using a stop word table customized for the automobile field to generate candidate word group sequences and count the occurrence frequencies of the candidate word groups; Identification of new explicit identity identification keywords based on the frequency distribution of the candidate word groups and an inter-class discriminability index, and dynamic updating of the new keywords to the non-real user identification rule library; Iterative execution of the preliminary filtering, semantic segmentation and keyword identification steps based on the updated non-real user identification rule library until no new explicit identity identification keywords are identified in a single complete iteration process; Output of the comment data retained after final filtering as a real user comment data set.

2. The data filtering method of automobile user reviews according to claim 1, characterized in that, The inter-class discriminability index is at least one of a TF-IDF value, an information gain or a chi-square statistic, and only when the inter-class discriminability index of a candidate word group in a suspicious comment subset exceeds a preset threshold, is it determined as a new explicit identity identification keyword.

3. The data filtering method of automobile user reviews according to claim 1, characterized in that, After obtaining the automobile user comment data, further comprising: obtaining the user's historical comment behavior data, if the same user publishes comments with similar content structure on more than three different automobile brands within a preset time window, the user is marked as a suspicious user, and the comment data thereof is preferentially excluded in the preliminary filtering.

4. The data filtering method of automobile user reviews according to claim 1, characterized in that, The semantic segmentation uses a word segmentation strategy combining automobile field dictionary matching and local context window to maintain the integrity of the "vehicle technical feature-user evaluation viewpoint" binary semantic unit.

5. The data filtering method of automobile user reviews according to claim 4, characterized in that, Further semantic consistency verification of the "vehicle technical feature-user evaluation viewpoint" binary semantic unit based on a pre-constructed automobile field knowledge graph, if there is no reasonable association path or fact conflict between the two in the knowledge graph, the credibility weight of the corresponding comment is reduced or the comment is excluded.

6. The data filtering method of automobile user reviews according to claim 1, characterized in that, Before outputting the real user comment data set, further comprising: calculating a comprehensive credibility score for each retained comment, the credibility score being determined based on at least two of the following: (1) whether the user has completed vehicle binding or real name authentication; (2) the number of specific vehicle model parameters or use scenario details contained in the comment content; (3) whether the length of the comment is in the typical interval of real users; (4) the logical reasonableness of the comment posting time and the vehicle delivery time; Only when the credibility score is higher than a preset threshold, the comment is included in the real user comment data set.

7. The data filtering method of automobile user reviews according to claim 1, characterized in that, The stop word table customized for the automobile field excludes automobile technical feature terms, performance evaluation words and user experience description words, and only retains grammatical function words and punctuation marks without substantive semantic contribution.

8. A data filtering system for car user reviews, characterized by, Comprise: A data acquisition unit configured to obtain automobile user comment data from heterogeneous data sources, the comment data comprising user identification, text comment content and additional labels; The preliminary filtering unit is configured to perform preliminary filtering on the comment data based on a preset non-real user identification rule library, and the non-real user identification rule library stores explicit identity keywords, and the explicit identity keywords include words associated with automobile sales services, commercial promotion content production, or competitor derogatory behaviors. The semantic segmentation unit is configured to perform semantic segmentation on the preliminary filtered comment data using a stop word table customized for the automobile field, generate candidate word group sequences, and count the occurrence frequencies of each candidate word group. The rule updating unit is configured to identify new explicit identity keywords based on the frequency distribution of the candidate word groups and the inter-class discriminability index, and dynamically update the new keywords to the non-real user identification rule library. The iteration control unit is configured to trigger the preliminary filtering unit, the semantic segmentation unit, and the rule updating unit to perform cyclic processing until no new explicit identity keywords are identified in a single complete iteration process. The data output unit is configured to output the comment data retained after final filtering as a real user comment data set.

9. An electronic device, comprising: The computer program is executed by the processor to implement the data filtering method of the automobile user comment according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the data filtering method of the automobile user comment according to any one of claims 1 to 7.

Images