System and method for generating content summary with minimal illusion

Abstract

Systems and methods are provided for generating a content summary with minimal illusion. In particular, the systems and methods may be applicable to EV charging stations. One or more natural language processing models (NLP models), such as aspect-based sentiment analysis models (ABSAs) and summary models, may be used to generate a summary of information associated with a charging station. A mechanism may also be used to ensure that the generated summary is accurate and does not include illusion.

Description

Technical Field

[0001] This disclosure relates to systems and methods for generating content overviews with minimal illusions. Background Technology

[0002] As more drivers rely on electric vehicles (EVs) for their daily transportation needs, the need for reliable and user-friendly information about EV charging locations becomes increasingly important. EV drivers depend heavily on customer reviews to choose suitable charging locations, but the sheer volume of reviews makes quickly identifying relevant insights challenging. Currently, users must sift through numerous reviews to find useful information on factors such as station functionality, charging speed, and the overall experience. This process is time-consuming and inefficient, especially when quick decisions are required.

[0003] The diverse nature of customer feedback further complicates the challenge. Reviews often cover different aspects such as usability, reliability, and convenience, making it challenging for users to gain a comprehensive understanding from a single review. Furthermore, the presence of both positive and negative reviews can be confusing, as users need to weigh differing opinions to form an overall judgment.

[0004] Additionally, the USA's public EV infrastructure encompasses a variety of Charge Point Operators (CPOs), each offering different mobile apps, features, and options. This diversity of services leads to unique issues at these charging stations that result in poor charging experiences. Fortunately, some previous users have provided advice and solutions for these specific problems. However, not all customers read these user reviews, and for those who do, the process can be very time-consuming.

[0005] Another challenge is the existence of multiple platforms for evaluation. EV drivers often consult various data sources to gather comprehensive information. This fragmented information increases the complexity of making informed choices, as drivers spend additional time cross-referencing evaluations, which undermines the convenience promised by EV technology.

[0006] Finally, as the EV market continues to expand, the volume of reviews will increase, exacerbating these issues. The need for an effective way to synthesize and present this information in a user-friendly manner becomes increasingly urgent. More importantly, this information can activate digital services that assist EV drivers in identifying optimal charging locations based on their personal preferences. Summary of the Invention

[0007] This disclosure describes systems and methods for generating content overviews with minimal illusions (e.g., incorrect information included within the overview). While this document describes a use case for generating an overview for EV charging stations, the methods described can also be applied to other use cases.

[0008] Users typically expect to see a concise overview including key information about areas of interest. However, such information is often not readily accessible in this concise format and requires users to obtain multiple pieces of information from various types of data sources. This results in a time-consuming and inefficient process, and in some cases, users may not be able to locate all the desired information. This typical process is particularly detrimental when users need information to make decisions within a short timeframe (e.g., if a user is driving and needs to identify the EV charging station to use, as described further below).

[0009] In the use case of EV charging stations, when users are deciding which charging station to use to charge their vehicles (e.g., if a user currently needs to charge their vehicle and is looking for a nearby charging station, or if a user is planning a future trip and expects to pre-plan their route to include charging stations with the attributes the user desires), users may expect to see concise information about key aspects of EV charging stations. However, typical sources of this type of information include user reviews posted on various types of online platforms, and users may find it difficult to navigate these data sources to efficiently find information about a variety of charging stations.

[0010] The system described in this paper addresses these challenges by generating concise and useful summaries of information from various data sources. Specifically, the system includes multiple Natural Language Processing (NLP) models. In some embodiments, the models may include an aspect-based sentiment analysis model (ABSA) and an overview model; however, any other number of models (and combinations of different types of models) may also be used. In some cases, a single model may also be used to perform all the tasks described herein. These models can be developed using large language models (LLMs) by employing both cue engineering and fine-tuning techniques.

[0011] Returning to the EV charging station use case, the system leverages NLP models to outline customer reviews (and / or any other relevant information) of EV charging locations. As a non-limiting example, by employing aspect-based sentiment analysis (ABSA), the system can generate detailed overviews across a variety of topics, such as functionality, charging speed, customer service, and convenience. This approach not only condenses information from diverse data sources into an easily understandable format but also ensures that users receive nuanced insights into specific aspects of their charging experience. Furthermore, the system can receive periodic snapshots (e.g., daily or any other time period) from each data provider. Therefore, the overviews generated by the system are based on up-to-date data from various data sources.

[0012] Regarding this use case, the system addresses the core challenges of information overload and fragmented feedback, enabling EV drivers to make informed decisions quickly and efficiently. The system enhances the user experience by providing clear, concise, and relevant information, thereby increasing overall EV driver satisfaction with the charging infrastructure. Furthermore, the system uses NLP models to analyze community reviews and identify common problems and their solutions. These insights are particularly valuable for new EV drivers using specific charging stations, enhancing their overall experience and ensuring a seamless charging process.

[0013] In some embodiments, the system can also be integrated into consumer-facing applications to provide users with enhanced decision-making capabilities as they select charging stations and determine the best charging solution for their vehicles. For example, users can leverage the application to not only search for available charging stations but also view an overview of charging stations generated by the system, allowing them to make more informed decisions.

[0014] To ensure the relevance of the overviews, the system can also incorporate various triggers for updates. For example, when a user uses the app to select and utilize charging locations, the app can prompt the user for feedback on the accuracy and usefulness of the overviews provided by the app. If the feedback confirms that the current overview is satisfactory, the overview is retained. However, if the feedback is unsatisfactory, the model (e.g., ABSA and / or the overview) can be improved to enhance the overview. As another example, the system can periodically update the overviews after receiving a certain threshold number of new comments. The overview job is scheduled to run periodically (e.g., daily, weekly, etc.), and new overviews can be inferred based on the number of new comments since the last overview. To ensure the temporal relevance of the comments, a threshold can be established, and only information within the threshold (e.g., information less than three months or any other time frame) can be considered. This continuous feedback loop ensures that the overviews evolve with the user experience and maintain their relevance and accuracy.

[0015] One challenge associated with generating overviews using these types of models is that the models often produce illusory information, meaning that they generate inaccurate or otherwise unbased information. Therefore, the system described in this paper not only provides a method for generating overviews of information in different formats and from different data sources into a concise and easily accessible format that can be viewed via an application's user interface, but also addresses the technical challenges associated with using models for such overview tasks. That is, the system reduces or eliminates the illusions produced by such models, ensuring that the information in the overviews is not only convenient but also accurate. At least regarding... Figures 3A-3B Specific enhancements to the model used to reduce or eliminate hallucinatory information are described.

[0016] Again, turning to the example of EV charging stations, the system offers several advantages over existing methods, significantly enhancing the EV charging experience by summarizing user reviews and other relevant information. Current methods require manual review of isolated and fragmented raw data in the form of user reviews available across multiple platforms. These reviews are often difficult to understand due to their fragmented nature and varied formats. In contrast, the system described in this paper provides users with quick and easy access to summarized reviews, highlighting the most relevant and critical information about the charging station. This reduces the time and effort required to sift through information, allowing users to make informed decisions quickly. By generating both overall and detailed overviews based on specific topics such as functionality, charging speed, and customer service using ABSA, users receive nuanced insights into all aspects of their charging experience. This detailed information helps users gain a comprehensive understanding of the strengths and weaknesses of each charging location. Using ABSA to identify sentences relevant to each topic, the overview model is combined, highlighting potential problems and previous customer recommendations. This approach helps users understand what to expect and make more informed decisions, ensuring a more reliable and seamless charging experience. Furthermore, the implementation of large language models (LLMs) and the fine-tuning techniques used for ABSA and overviews ensure that the solution can efficiently handle the ever-increasing volume of evaluations. This capability allows the system to scale with the number of users and charging stations, thus continuously improving as more data becomes available. Moreover, the adaptability of these models ensures they remain effective as new patterns and topics emerge in customer feedback, making the solution robust and sustainable.

[0017] These and other advantages of this disclosure are provided in detail herein. Attached Figure Description

[0018] Specific embodiments are illustrated with reference to the accompanying drawings. The same reference numerals may be used to indicate similar or identical items. Various embodiments may utilize elements and / or components other than those shown in the drawings, and some elements and / or components may not be present in various embodiments. Elements and / or components in the drawings are not necessarily drawn to scale. Throughout this disclosure, singular and plural terms may be used interchangeably depending on the context.

[0019] Figure 1 A flowchart is depicted outlining an overview of generating content with minimal illusions according to this disclosure.

[0020] Figures 2A-2B A flowchart for model training according to this disclosure is depicted.

[0021] Figures 3A-3B A flowchart is depicted for a feedback mechanism for mitigating model hallucinations according to this disclosure.

[0022] Figure 4 An exemplary overview process according to this disclosure is described.

[0023] Figure 5 An exemplary topic-level overview according to this disclosure is described.

[0024] Figure 6 The environment in which the technologies and structures for providing the systems and methods disclosed herein can be implemented is described.

[0025] Figure 7 A flowchart of a method for predicting vehicle navigation according to this disclosure is depicted. Detailed Implementation

[0026] The present disclosure will be described more fully below with reference to the accompanying drawings, which illustrate exemplary embodiments of the present disclosure and are not intended to be limiting.

[0027] Figure 1 A flowchart 100 is depicted to illustrate an overview of content generated with minimal illusion. Flowchart 100 depicts some of the exemplary operations that can be performed by a system as described herein. However, flowchart 100 is not intended to be limiting in any way, and the system can perform more than... Figure 1 The operations shown can be more or fewer than the operations shown.

[0028] Flowchart 100 illustrates a master database 102, which may include any of the data obtained from various data sources 104 for generating the data outlined herein. Flowchart 100 also illustrates that data may be periodically obtained from the various data sources 104 to update the data stored in the master database 102. The master database 102 may also be updated in real time with data from the data sources 104.

[0029] Initially, the overview process may optionally involve performing preprocessing on any of the data stored in the main database 102 at operation 106. For example, for the EV charging station use case, the data stored in the main database 102 may include user reviews obtained from various data sources 104. However, some of the user reviews may be provided in formats that are difficult for the model (e.g., ABSA model 114 and / or overview model 120) to process. In some cases, the model may be able to process all the data; however, some of the data may not include useful information (or may include incorrect information), and the model may not expect to use that data. For example, erroneous user reviews may not include any information, or user reviews may include certain non-alphanumeric characters or other types of symbols. In these examples, preprocessing may involve removing erroneous user reviews and / or removing certain non-alphanumeric characters or other types of symbols from the main database 102. However, any other type of data modification (including data removal) may be made as part of the preprocessing.

[0030] Once the data has been preprocessed (if preprocessing is performed), the processed data 108 can be used to generate an overview for all locations. The overview process can be performed as follows. ABSA model 114 can receive the processed data (continuing with the exemplary use case, ABSA model 114 can receive user reviews and / or any other data related to the charging stations). At operation 116, the ABSA model can use the data to perform predictions. Specifically, ABSA model 114 can process the data to identify portions of the data useful for generating the overview. ABSA model 114 can also classify the data. For example, as described elsewhere herein, predetermined categories of information that may be relevant to users can be established, and ABSA model 114 can determine whether the data is associated with any of these predetermined categories (examples of these categories are shown in Table 1, which is provided below; however, these examples are not intended to be limiting in any way).

[0031] Table 1

[0032]

[0033]

[0034]

[0035]

[0036] ABSA model 114 can also determine the “sentiment” of data or parts of data. For example, ABSA model 114 can determine whether each sentence within a given user review is “positive,” “negative,” or “neutral” (however, other sentiment classifications may also be used). The overall sentiment of a user review can also be determined based on the sentiment assigned to each of the sentences that make up the whole user review. ABSA model 114 can output these predictions (and / or any other relevant information) as first output data 118.

[0037] Once the ABSA model 114 generates first output data 118, the first output data 118 and the input data can be provided to the overview model 120. The overview model 120 can then use this information to generate one or more overviews that can be presented to a user (e.g., via an application's user interface). In some embodiments, one or more overviews may include a comprehensive overview 122, a topic-level overview 124, and a user tip and suggestion overview 126. These types of overviews are merely exemplary, and the overview model 120 may also provide other types of overviews.

[0038] For example, the Comprehensive Overview 122 provides a holistic view of all user feedback related to a specific EV charging location. The Comprehensive Overview 122 combines key points of interest and concern from all reviews, giving users a quick snapshot of the station's overall performance and user satisfaction. This Comprehensive Overview 122 helps users immediately understand what to expect without delving into the details. As an example of the Comprehensive Overview 122, most reviews highlight a mixed experience of functionality and reliability. Some users encountered issues with payment methods, connection errors, and charging interruptions. Others reported seamless and reliable charging sessions. Most reviews indicate that the availability and accessibility of EV charging stations are generally positive. Most users found all stations online and available, but some experienced initial connection problems. There is a mixed feedback regarding charging speed and efficiency; some users reported fast charging speeds, while others noted slower rates and discrepancies between billed kWh and received kWh. Most users found the charging station locations convenient. Users appreciated clean, well-lit stations with nearby convenience stores. There is a mixed experience with customer service. Some users had difficulty contacting customer service or resolving issues, while others found the staff friendly and helpful.

[0039] For example, topic-level overviews can focus more on specific areas of interest. These overviews break down general feedback into categorized topics such as functionality and reliability, charging speed and efficiency, and customer service. Each overview can capture the essence of user sentiment within that topic, allowing users to understand the detailed aspects of the charging experience most relevant to their needs.

[0040] For example, the User Tips and Suggestions Overview 126 can compile useful tips and suggestions provided by users, such as specific issues to be aware of at certain sites. The User Tips and Suggestions Overview 126 can provide actionable advice from the user community, thereby enhancing the usefulness of the overview by incorporating peer wisdom and recommendations.

[0041] These are merely an overview of the exemplary types that can be generated, and overviews of any other types can also be generated.

[0042] In addition, the system can receive feedback on the overview features displayed via the customer-facing application 130. If the feedback is deemed unsatisfactory and the user indicates a need for enhancement, the system can trigger a process to improve the model and recalculate the overview.

[0043] Even after the overview model 120 generates the initial overview, the system can generate an updated overview based on new information received. In some cases, the system can automatically process any new information received, allowing the overview to be updated in real time. However, in other cases (as shown in flowchart 100), the system can periodically update the overview only based on the amount of new information already received. For example, flowchart 100 shows condition 112, which involves determining whether the amount of new information exceeds a defined threshold. In this exemplary implementation, if the number of new comments for a location since the last overview exceeds the threshold, ABSA model 114 and overview model 120 can update any overview. This ensures that the overview is not static but continuously enhanced and updated. In the use case of EV charging stations, this dynamic approach allows EV drivers to always have access to the most relevant and up-to-date information, thereby enhancing their decision-making process and overall satisfaction with the charging infrastructure.

[0044] Figures 2A-2B A flowchart 200 for model training is depicted. Initially, cue engineering techniques are employed to predict ground truth for training samples, saving significant time compared to labeling data from scratch. Subsequently, the system evaluates and corrects the ground truth data (or this process can be performed manually by the user). The labeled data is used to fine-tune a large language model to achieve good performance in the EV domain space (or other applicable domain spaces).

[0045] Figure 3A A flowchart 300 depicts a feedback mechanism for mitigating model illusion. Evaluating an overview using a large language model (LLM) is challenging due to the difficulties in providing ground reality, making it difficult to consistently measure the quality of the overview. To address these issues, an iterative self-improving algorithm with feedback loops is implemented. In some embodiments, flowchart 300 may include an overview process 302 and an evaluation and feedback process 310.

[0046] Beginning with the overview process 302, input data 304 may be received. For example, in the use case of an EV charging station, input data 304 may include user reviews of the charging station (and / or any other relevant information). Operation 306 involves generating an overview 308 of the charging station using the input data 304. For example, overview 308 may be generated from an overview model as described herein.

[0047] Moving to the evaluation and feedback process 310, the overview 308 and input information 304 can be provided to a "question-answer generation model" (which can be, for example, a large language model or any other suitable model of any type) to generate one or more questions related to the information included in the input data 304 and the overview 308 (in operation 310). The question-answer generation model can also generate a first set of answers based on the content of the input data 304 and a second set of answers based on the overview 308. Figure 3B The document shows examples of questions and answers generated based on the overview.

[0048] Once the question and answer sets have been generated using the question and answer model, condition 312 involves determining whether there are differences between the information included in the first and second answer sets. In some cases, the answer included in each of the two answer sets can be either a "yes" or a "no" answer, and the difference determination may involve determining whether the answer to a question is "yes" to one answer set and whether the answer to that question is "no" to the other answer set.

[0049] although Figure 3B The example shown illustrates "yes" and "no" answers, but this is merely illustrative, and the answers are not necessarily limited to only "yes" and "no" answers. If the question type is expanded to include more descriptive responses, a semantic similarity metric (such as BERTScore (or any other suitable technique)) can be implemented to compare the answers. This would allow the system to assess whether two answers are substantially similar, even if their wording differs. Using BERTScore or similarity methods would allow the model to focus on the semantic content of the response and ensure that the generated answers are meaningfully aligned, rather than relying solely on exact phrases.

[0050] Then, condition 314 involves determining whether the questions generated by the question-and-answer model are topic-related. In some embodiments, a large language model can be used in a self-feedback loop to determine whether a question is topic-related. The model can be prompted to evaluate whether the generated questions are relevant to a specific topic. This can be achieved with carefully refined prompts that ask the model whether the question is aligned with a given topic.

[0051] If discrepancies are detected, the questions are refined at operation 318, and their relevance to the topic is checked again. If a question is irrelevant to the topic, feedback is provided to ensure proper alignment. This iterative approach allows for continuous refinement and enhancement of the overview and evaluation process, reducing illusions and contradictory information in the overview results, thereby enhancing both accuracy and consistency.

[0052] Figure 3B It shows about Figure 3A An example of the hallucination mitigation process described is given. The example shows an initial overview 330 generated by an overview model for EV charging stations. Using the initial overview 330, a question and answer model generates one or more questions 332 and a first set of answers 334 and a second set of answers 336 to the questions. For example, Figure 3B Five different questions are shown, generated by the question and answer model based on the initial overview 330 and the input data used to generate the initial overview 330. Figure 3B Two sets of five different answers for each of the five questions are also shown, where the first set of answers 334 is based on the input information, and the second set of answers 336 is based on the information included in the initial overview 330. At operation 334, the system determines that there is a difference between the answers to the third question included in the first set of answers 334 and the answers to the third question included in the second set of answers 336.

[0053] At operation 334, the system determines that the third question is relevant to the topic. Therefore, operation 336 involves refining the third question. For example, Figure 3B The third question is shown to be improved from "Is Charger 1 generally reliably but often has longwaits?" to "Is Charger 1 reliable but often associated with long wait times according to user reviews?". Then, a question-and-answer model is used to generate updated answers based on the improved question. At operation 338, a difference is still identified between the answers to the third question included in the first answer set 334 and the answers to the third question included in the second answer set 336. This persistent difference is indicated to the overview model, and the overview model corrects the initial overview 330 to a revised overview 340. The revised overview adjusts the highlighted sentence shown in the initial overview 330 to "Charger 1 is generally reliable" in the revised overview 340 to provide more accurate information.

[0054] Figure 4 An exemplary overview process 400 is described. Using an NLP model, two types of overviews can be generated: (1) a general overview and (2) a detailed overview by topic. The general overview provides a high-level view of user feedback, capturing general sentiment and key points from the collected reviews. For example, the general overview might highlight common problems such as frequent power outages, charger malfunctions, and customer service issues, as well as positive aspects such as charging speed and station convenience. The detailed overview by topic explores more deeply specific aspects identified by ABSA, such as functionality, reliability, charging speed, customer service, and convenience. By summarizing reviews related to each topic separately, the system ensures that users have a comprehensive understanding of their previous charging experiences in each key aspect.

[0055] ABSA can identify the overall sentiment for each topic for each location. For example, there might be a set of predefined sentiment categories (e.g., positive, negative, or neutral), and ABSA can categorize all portions of a user review into one of these predefined sentiments. However, in some cases, ABSA may not need to rely on predefined categories and can instead generate non-predefined sentiment classifications. In one exemplary implementation, ABSA can classify each sentence (or part of a sentence) of a user review. In this implementation, the overall sentiment of a user review can be determined by comparing each of the sentiment categories to a threshold percentage. For example, if more than 70% of the sentences for a particular topic are classified as "positive," the system can consider that topic to have overall positive feedback (similar logic can be applied to negative and neutral sentiments). Alternatively, the system can determine which of the sentiment categories is more prevalent. For example, if ABSA determines that three sentences are "positive" and one sentence is "neutral," the system can determine that the user review is overall positive.

[0056] Additionally, the system can apply extra weight to user feedback after viewing the system-generated overview. This weighted sentiment is then updated as new comments are added, ensuring that the sentiment reflects the most current user experience.

[0057] Figure 5 An exemplary overview 500 generated by a system as described herein is shown. Figure 5This illustrates how overviews can be generated for different types of attributes relevant to a use case. In the example of an EV charging station, some examples of such attributes could be functionality and reliability, charging speed and efficiency, and location and amenities. Other attributes can also be outlined. In some cases, the attributes outlined for a given use case can be predefined, and the same type of attributes can be outlined for all users. However, users can have the ability to manually indicate which attributes they most desire, and the overview presented to them via the user interface can include only those indicated attributes. In some cases, the system can also automatically determine which attributes are most relevant to the user without requiring manual instructions.

[0058] The system can also utilize user feedback to enhance the accuracy of the output generated by the model. EV drivers often encounter unique challenges at charging stations, which can exacerbate their range anxiety and hinder their ability to effectively charge their vehicles. In such cases, drivers typically seek assistance from customer support. However, if customer support is unresponsive, drivers may face difficulties completing the charging process.

[0059] As a first example, users face issues activating the charger due to the lack of a physical FLO card or a mobile app associated with the charging station. This problem could be resolved if a previous user who experienced a similar challenge provided a detailed review of how to activate the charger by adding a physical card using the mobile app.

[0060] As a second example, one of the charging stations might be experiencing a problem and not accepting card payments, only accepting payments via a mobile app. This could be resolved if previous users suggested tapping the card on the payment device for 5-10 seconds.

[0061] To address these common challenges faced by users, the system focuses on extracting key user suggestions using a combination of ABSA and an overview model tailored to user tips and suggestions. Specifically, the ABSA model identifies and filters all reviews categorized under user tips and suggestions. This step aims to isolate user-generated content that provides valuable insights and solutions. The overview model then compiles and summarizes the extracted reviews. This model plays a crucial role in synthesizing user suggestions into actionable recommendations tailored to the user. By leveraging these models, the solution aims to empower users by providing practical solutions to common problems encountered at charging stations. This approach enhances the user experience and helps users overcome challenges, ultimately contributing to a more seamless and efficient charging process.

[0062] Figure 6A block diagram of a system 600 for predicting vehicle navigation according to this disclosure is depicted. System 600 may include one or more vehicles 601, one or more user devices 602, infrastructure 603, and one or more servers 604 (or servers 605) communicatively coupled to each other via one or more networks 606 (or networks 606). System 600 is not necessarily intended to be exhaustive, but only illustrates exemplary components that may be included in the exemplary system 600. Any reference to a single element (e.g., “server 604”, etc.) may similarly refer to any other number of such elements. Similarly, references to multiple such elements may also refer to a single element.

[0063] Vehicle 601 and / or the driver shall perform and / or execute the actions described herein in accordance with the owner's manual and safety guidelines. Furthermore, any actions taken by the driver based on notifications / alarms provided by vehicle 102 shall comply with all rules specific to the location and operation of vehicle 601 (e.g., federal, state, national, city, etc.). Notifications / alarms provided by vehicle 601 shall be considered as recommendations and shall be followed only in accordance with any rules specific to the location and operation of vehicle 601.

[0064] User device 602 may be associated with a user and may be, for example, a mobile phone, laptop computer, computer, tablet computer, wearable device, or any other similar device with communication capabilities. User device 602 may include application 620 that presents a user interface to the user. Application 620 may present an overview generated by overview model 636 as described herein. Additionally, in the use case of EV charging stations, application 620 (or another application not shown in the figures) may be used by the same or other users to submit user reviews about a particular charging station. These user reviews can then be used as input data processed by server 604 to generate an overview.

[0065] Server 604 can be configured to receive data from one or more vehicles 601, user devices 602, infrastructure 603, and / or any other device configured to capture location-related data. Server 604 can also be configured to process any data as described herein. For example, server 604 can host any of the models used for overview generation, hallucination mitigation, and any other related tasks as described herein. As mentioned above, some or all of the processing can also be performed by one or more vehicles 601, user devices 602, etc. Therefore, one or more machine learning models can also (or alternatively) be hosted on one or more vehicles 601, user devices 602, etc. Thus, server 604 can include an ABSA model 634, an overview model 636, and a hallucination mitigation module 638, which can utilize the question and answer models as described above.

[0066] Network 606 illustrates an exemplary communication infrastructure in which connected devices discussed in various embodiments of this disclosure can communicate. Network 606 may be and / or include the Internet, a private network, a public network, or other configurations operating using any one or more known communication protocols such as Transmission Control Protocol / Internet Protocol (TCP / IP), Bluetooth, etc. ® BLE, Wi-Fi based on the IEEE standard 802.11, Ultra Wideband (UWB), and cellular technologies such as Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), High-Speed ​​Packet Access (HSPDA), Long Term Evolution (LTE), Global System for Mobile Communications (GSM), and 5G, to name just a few.

[0067] Any components of system 600 can also form a mesh network. This can be advantageous in scenarios where a particular device is not connected to a wide area network and cannot transmit data over long distances (e.g., from a vehicle to server 604). In such scenarios, devices unable to perform long-distance communication may still be able to perform short-range communication with other nearby devices. For example, traffic infrastructure may capture an image of a location but may not be able to transmit the data to server 604. Instead, the traffic infrastructure can transmit the image to a nearby vehicle, which can then transmit the data to server 604. Such data transmission can also be performed between any other type of device.

[0068] In some aspects, user device 602 may be configured to connect to vehicle computer 608 and / or system 612 via network 606, which may communicate via one or more wireless connections, and / or the user device may communicate using Near Field Communication (NFC) protocol, Bluetooth, etc. ® Protocols, Wi-Fi, Ultra-Wideband (UWB), and other possible data connectivity and sharing technologies are used to directly connect to one or more vehicles 601.

[0069] Any of the components of system 600 may further include one or more processors (e.g., processors 610, 630, etc.) configured to communicate with one or more memory devices configured to communicate with a corresponding computing system (e.g., memory 612, memory 632, etc.). The processors may utilize the memory to store programs in code and / or store data to execute aspects of this disclosure. The memory may be a non-transitory computer-readable storage medium or memory that stores interface management program code. The memory may include any or a combination of volatile memory elements (e.g., dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), etc.) and may include any one or more non-volatile memory elements (e.g., erasable programmable read-only memory (EPROM), flash memory, electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), etc.). Although only vehicle 601 and server 604 are shown as having processors and memory, this is for illustrative purposes only, and any other component of the system (such as user device 602 and infrastructure 603) may also include processors and memory.

[0070] In some aspects, vehicle 601 may also include an infotainment system 614 (or a vehicle human-machine interface (HMI)). The infotainment system 614 may include a touchscreen interface portion and may include voice recognition features and biometric identification capabilities, which may identify the user based on facial recognition, voice recognition, fingerprint identification, or other biometric means. In other aspects, the infotainment system 614 may be further configured to receive user commands via the touchscreen interface portion and / or output or display notifications, navigation maps, etc., on the touchscreen interface portion. Similar to application 620 of user device 602, the infotainment system 614 may present a generated overview to the user. That is, the user can view the overview via user device 602 and / or vehicle 601. The information may also be presented to the user in other ways. Vehicle 601 may also include a sensor system 616, which may include any number of sensors of different types.

[0071] Figure 7An exemplary method 700 is illustrated. Method 700 begins at step 702. At step 702, method 700 may include receiving first input data from one or more data sources at a first time, the first input data including first user reviews of one or more electric vehicle (EV) charging stations. At step 704, method 700 may include causing a first model to predict first output data, the first output data including a first category associated with the first input data. At step 706, method 700 may include causing a second model to generate one or more first overviews based on the first input data and the first output data. At step 708, method 700 may include causing a third model to determine that one or more first overviews include illusions generated by the second model. At step 710, method 700 may include causing a second model to generate one or more second overviews based on the determination that one or more first overviews include illusions generated by the second model. At step 712, method 700 may include causing one or more second overviews to be presented via a user interface of the device.

[0072] In the foregoing disclosure, reference has been made to the accompanying drawings, which form a part of the foregoing disclosure, illustrating specific embodiments in which the present disclosure may be practiced. It should be understood that other implementations and structural changes may be made without departing from the scope of the present disclosure. References to "an embodiment," "embodiment," "example embodiment," etc., in this specification indicate that the described embodiment may include specific features, structures, or characteristics, but each embodiment may not necessarily include said specific features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when features, structures, or characteristics are described in connection with embodiments, those skilled in the art will recognize such features, structures, or characteristics in conjunction with other embodiments, whether explicitly described or not.

[0073] Furthermore, where appropriate, the functions described herein may be performed by one or more of the following: hardware, software, firmware, digital components, or analog components. For example, one or more application-specific integrated circuits (ASICs) may be programmed to perform one or more of the systems and programs described herein. Certain terms are used throughout the specification and claims to refer to specific system components. As those skilled in the art will appreciate, components may be referred to by different names. This document is not intended to distinguish between components with different names but identical functions.

[0074] It should also be understood that the term "example" as used herein is intended to be non-exclusive and non-restrictive in nature. More specifically, the term "example" as used herein refers to one of several examples, and it should be understood that there is no undue emphasis or preference on the particular example described.

[0075] Computer-readable media (also known as processor-readable media) include any non-transitory (e.g., tangible) medium that contributes to providing data (e.g., instructions) that can be read by a computer (e.g., by the computer's processor). Such media can take many forms, including but not limited to non-volatile and volatile media. Computing devices may include computer-executable instructions, wherein the instructions may be executable by one or more computing devices (such as those listed above) and stored on a computer-readable medium.

[0076] Regarding the processes, systems, methods, heuristics, etc., described herein, it should be understood that although the steps of such processes, etc., are described as occurring in a certain ordered order, such processes can be practiced by performing the described steps in a different order than that described herein. It should also be understood that some steps may be performed simultaneously, other steps may be added, or some steps described herein may be omitted. In other words, the description of processes herein is provided for the purpose of illustrating various embodiments and should in no way be construed as limiting the claims.

[0077] Therefore, it should be understood that the above description is intended to be illustrative rather than restrictive. Many embodiments and applications beyond the examples provided will become apparent upon reading the above description. The scope should not be determined by reference to the above description, but rather by reference to the appended claims and the full scope of their equivalents. It is anticipated and expected that the techniques discussed herein will evolve in the future, and the disclosed systems and methods will be incorporated into such future embodiments. In conclusion, it should be understood that modifications and changes are possible with this application.

[0078] Unless explicitly indicated otherwise herein, all terms used in the claims are intended to be given their ordinary meaning as understood by one skilled in the art as described herein. Specifically, unless the claims explicitly limit the recitation to the contrary, the use of singular articles such as “a,” “the,” or “the” should be interpreted as one or more of the elements indicated by the recitation. Unless otherwise specifically stated or otherwise understood in the context of use, conditional language such as, in particular, “can,” “may,” “may,” or “may” is generally intended to express that some embodiments may include certain features, elements, and / or steps, while other embodiments may not include certain features, elements, and / or steps. Therefore, such conditional language is generally not intended to imply that one or more embodiments require each feature, element, and / or step in any way.

[0079] According to an embodiment, the first model is an aspect-based sentiment analysis (ABSA) model, and the second model is an overview model.

[0080] According to an embodiment, the computer-executable instructions further cause the one or more processors to perform the following operations: causing the third model to generate one or more first questions using the first input data and the one or more first overviews; causing the third model to generate one or more first answers to the one or more first questions based on the first input data, and to generate one or more second answers to the one or more first questions based on the one or more first overviews; and determining the differences between the one or more first answers and the one or more second answers.

[0081] According to an embodiment, determining that the one or more first overviews include hallucinations further includes: causing the third model to refine the one or more first questions to generate one or more second questions; causing the third model to generate one or more third answers to the one or more second questions based on the first input data, and to generate one or more second answers to the one or more second questions based on the one or more first overviews; and determining a second difference between the one or more first answers and the one or more second answers, wherein causing the second model to generate the one or more second overviews is based on the second difference.

[0082] According to an embodiment, the computer-executable instructions further cause the one or more processors to perform the following operations: preprocess the first input data to perform at least one of the following: remove empty user reviews or modify the format of user reviews in the user reviews.

[0083] According to an embodiment, the computer-executable instructions further cause the one or more processors to perform the following operations: receiving second input data from the one or more data sources at a second time, the second input data including second user reviews of one or more electric vehicle (EV) charging stations; determining that the second input data includes a threshold number of user reviews; and based on the determination that the second input data includes a threshold number of user reviews, causing the second model to update the one or more second overviews based on the first input data and the first output data.

Claims

1. A system comprising: Memory, the memory storing computer-executable instructions; as well as One or more processors, the one or more processors being configured to access the memory and execute the computer-executable instructions to: Receive first input data from one or more data sources at the first time, the first input data including first user reviews of one or more electric vehicle (EV) charging stations; The first model is made to predict first output data, the first output data including a first category associated with the first input data; The second model generates one or more first summaries based on the first input data and the first output data; The third model determines that the one or more first overviews include hallucinations generated by the second model; Based on the determination that the one or more first overviews include hallucinations generated by the second model, the second model generates one or more second overviews; as well as The second overview is displayed via the device's user interface.

2. The system of claim 1, wherein the first model is an aspect-based sentiment analysis (ABSA) model and the second model is an overview model.

3. The system of claim 1, wherein determining that the one or more first overviews include hallucinations further includes: The third model uses the first input data and the one or more first overviews to generate one or more first questions; The third model generates one or more first answers to the one or more first questions based on the first input data, and generates one or more second answers to the one or more first questions based on the one or more first summaries; as well as Determine the differences between the one or more first answers and the one or more second answers.

4. The system of claim 3, wherein determining that the one or more first overviews include hallucinations further includes: The third model improves the one or more first problems to generate one or more second problems; The third model generates one or more third answers to the one or more second questions based on the first input data, and generates one or more second answers to the one or more second questions based on the one or more first overviews; as well as Determine a second difference between the one or more first answers and the one or more second answers, wherein the second model generates the one or more second summaries based on the second difference.

5. The system of claim 1, wherein the one or more processors are further configured to execute the computer-executable instructions to: The first input data is preprocessed to perform at least one of the following: remove empty user reviews or modify the format of user reviews in the user reviews.

6. The system of claim 1, wherein the one or more processors are further configured to execute the computer-executable instructions to: At a second time, second input data is received from the one or more data sources, the second input data including second user reviews of one or more electric vehicle (EV) charging stations; The second input data is determined to include a threshold number of user reviews; as well as Based on determining that the second input data includes a threshold number of user reviews, the second model updates the one or more second overviews based on the first input data and the first output data.

7. The system of claim 1, wherein the first category includes at least one of the following: functionality and reliability, accessibility and availability, location, price, convenience, user tips, charging speed and efficiency, customer service, or charging station safety, wherein the first output data further includes the sentiment of the user rating in the first user rating, the sentiment including at least one of the following: positive, negative, or neutral.

8. A method comprising: Receive first input data from one or more data sources at the first time, the first input data including first user reviews of one or more electric vehicle (EV) charging stations; The first model is made to predict first output data, the first output data including a first category associated with the first input data; The second model generates one or more first summaries based on the first input data and the first output data; The third model determines that the one or more first overviews include hallucinations generated by the second model; Based on the determination that the one or more first overviews include hallucinations generated by the second model, the second model generates one or more second overviews; as well as The second overview is displayed via the device's user interface.

9. The method of claim 8, wherein the first model is an aspect-based sentiment analysis (ABSA) model, and the second model is an overview model.

10. The method of claim 8, further comprising: The third model uses the first input data and the one or more first overviews to generate one or more first questions; The third model generates one or more first answers to the one or more first questions based on the first input data, and generates one or more second answers to the one or more first questions based on the one or more first summaries; as well as Determine the differences between the one or more first answers and the one or more second answers.

11. The method of claim 10, wherein determining that the one or more first overviews include hallucinations further comprises: The third model improves the one or more first problems to generate one or more second problems; The third model generates one or more third answers to the one or more second questions based on the first input data, and generates one or more second answers to the one or more second questions based on the one or more first overviews; as well as Determine a second difference between the one or more first answers and the one or more second answers, wherein the second model generates the one or more second summaries based on the second difference.

12. The method of claim 8, further comprising: The first input data is preprocessed to perform at least one of the following: remove empty user reviews or modify the format of user reviews in the user reviews.

13. The method of claim 8, further comprising: At a second time, second input data is received from the one or more data sources, the second input data including second user reviews of one or more electric vehicle (EV) charging stations; The second input data is determined to include a threshold number of user reviews; as well as Based on determining that the second input data includes a threshold number of user reviews, the second model updates the one or more second overviews based on the first input data and the first output data.

14. The method of claim 8, wherein the first category includes at least one of: functionality and reliability, accessibility and availability, location, price, convenience, user tips, charging speed and efficiency, customer service, or charging station safety, wherein the first output data further includes the sentiment of the user reviews in the first user reviews, the sentiment including at least one of: positive, negative, or neutral.

15. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the following operations: Receive first input data from one or more data sources at the first time, the first input data including first user reviews of one or more electric vehicle (EV) charging stations; The first model is made to predict first output data, the first output data including a first category associated with the first input data; The second model generates one or more first summaries based on the first input data and the first output data; The third model determines that the one or more first overviews include hallucinations generated by the second model; Based on the determination that the one or more first overviews include hallucinations generated by the second model, the second model generates one or more second overviews; as well as The second overview is displayed via the device's user interface.

Images