Artificial intelligence tool for sustainability strategy

Abstract

Methods and systems for assessing sustainability of organizational practices with artificial intelligence (AI) tools are provided. Embodiments process textual data collected from a user, either through written documents or conversational interviews with an AI tool, to determine the practices of an organization in view of sustainability. Data is collected by decomposing text into excerpts and using generative AI tools to create corresponding vector embeddings. These vector embeddings are then concurrently searched for similarity to questions targeted to different aspects of sustainability. The generative AI tool is used to summarize findings, create comparisons to best practices and the practices of other organizations, and automate reporting. Schema for data collection, analysis, and reporting can be tailored according to input from subject-matter experts.

Description

ARTIFICIAL INTELLIGENCE TOOL FOR SUSTAINABILITY STRATEGYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Provisional Application 63 / 706,198 filed October 11, 2024 and entitled “ARTIFICIAL INTELLIGENCE TOOL FOR SUSTAINABILITY STRATEGY”, the contents of which are incorporated hereby by reference.FIELD OF THE INVENTION

[0002] The present application pertains to text processing and in particular to methods and systems for textual data collection, analysis, and reporting for sustainability assessment.BACKGROUND

[0003] Understanding the current sustainability of an organization, activity, or group can provide information on how it can improve to become more efficient, productive, fair, and ethical. However, sustainability can encompass many factors and span aspects such as environmental impact, resource consumption and waste production, human rights protection, team member representation and participation, risk management, stakeholder engagement, and other areas. Compiling, reviewing, and aggregating information from competitor reports, regulatory documents, and industry papers on best practices for sustainability can be time-consuming, biased, and unthorough. Likewise, comparison of current practices for an organization with best practices can be susceptible to similar issues. Conclusions drawn from such manual approaches may further not provide the most effective recommendations for improving the sustainability profile of an organization. Consultants are often hired to assess sustainability, but these can be costly and inaccurate and may still require significant time commitments from the organization.

[0004] Artificial intelligence (Al) technologies, such as large language models (LLMs), can be used to search report documents, such as for sustainability best practices. LLMs, in particular, canbe trained to process natural language text such that they can respond to specific queries for information in a given document. However, the currently available technologies often return inaccurate, irrelevant, or unreliable results, especially when responding to open-ended queries. They can further fail to reliably infer connections between multiple sources of information, which can vary greatly in scope and topic, nor are they able to effectively support long-duration information gathering sessions. For analysing an organization’s practices, current LLM technologies are not conducive to incorporating rules of analysis or embedding the expertise of subject matter experts (SMEs). They are still further ineffective in summarizing their conclusions cohesively and with justification.

[0005] Therefore, there is a need for methods and systems for Al tools for assessing sustainability that obviate or mitigate one or more limitations of the prior art.

[0006] This background information is provided to reveal information believed by the applicant to be of possible relevance to the present invention. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present invention.SUMMARY

[0007] An object of embodiments of the present disclosure is to provide methods and systems for automating sustainability assessments with Al tools.

[0008] A first aspect of the present disclosure is to provide a method for performing an assessment based on documents providing practices towards sustainability. The method can be performed at a computer device including a processor coupled to tangible, non-transitory processor-readable memory, with the computer device being coupled to a user interface and a plurality of databases. The method can comprise receiving, by the user interface one or more first user inputs providing a corresponding one or more document files each defining respective text representing a plurality of sustainability practices, and one or more second user inputs defining an assessment task. One or more sustainability aspects can then be retrieved from one database of the plurality of databases and a set of actions can be performed for each document file of the one or more document files.The set of actions can include: decomposing the text of the respective document file into a respective plurality of excerpts; calculating, by a generative Al model for each one of the respective plurality of excerpts, a respective vector embedding; and storing, at a further one of the plurality of databases, each vector embedding calculated for the respective document file. The further one database can then be searched in accordance with one of the one or more sustainability aspects to determine one or more relevant excerpts from among the respective plurality of excerpts of each document file, the respective vector embedding of each relevant excerpt having a certain similarity to a vector embedding of the sustainability aspect. The assessment task can then be completed, by the generative Al model, in accordance with the one or more relevant excerpts.

[0009] A second aspect of the present disclosure is to provide a method for performing an assessment based on interview responses towards sustainability. The method can be performed at a computer device including a processor coupled to tangible, non-transitory processor-readable memory, with the computer device being coupled to a user interface and a plurality of databases. The method can comprise: creating, by a generative artificial intelligence (Al) model, a question about one or more aspects of sustainability; providing, by the user interface, the question; receiving, by the user interface, a user answer in response to the question; retrieving, from one of the plurality of databases, one or more sustainability best practices; creating, by the generative Al model, one or more sustainability recommendations in accordance with the user answer and the one or more sustainability best practices; and providing, by the user interface, the one or more sustainability recommendations.

[0010] A third aspect of the present disclosure is to provide a method for verifying the accuracy of Al-generated response. The method can be performed at a computer device including a processor coupled to tangible, non-transitory processor-readable memory. The method can comprise: identifying an original response generated by a first generative Al model; selecting a second generative Al model, wherein the second generative Al model is different from the first generative Al model; reconstituting a context associated with the generation of the original response, the context comprising user inputs and external data utilized by the first generative Al model; performing, by the second generative Al model, a structured accuracy assessment of the original response utilizing the reconstituted context and pre-determined instructional prompts; andgenerating a structured assessment output. The structured assessment output can include an accuracy classification, such as Accurate, Incomplete, or Inaccurate, and a confidence score. In some embodiments, if the classification is Inaccurate, the structured assessment output can further comprise identification of inaccurate information, reasons for the inaccuracies, and a corrected response generated by the second generative Al model.

[0011] Embodiments have been described above in conjunctions with aspects of the present invention upon which they can be implemented. Those skilled in the art will appreciate that embodiments may be implemented in conjunction with the aspect with which they are described, but may also be implemented with other embodiments of that aspect. When embodiments are mutually exclusive, or are otherwise incompatible with each other, it will be apparent to those skilled in the art. Some embodiments may be described in relation to one aspect, but may also be applicable to other aspects, as will be apparent to those of skill in the art.BRIEF DESCRIPTION OF THE FIGURES

[0012] Further features and advantages of the present invention will become apparent from the following detailed description, taken in combination with the appended drawings, in which:

[0013] FIG. 1 shows an example of a call-flow for a competitivity analysis of practices for sustainability, according to an embodiment of the present disclosure.

[0014] FIG. 2 shows an example of a call-flow for a document chat for a sustainability assessment, according to an embodiment of the present disclosure.

[0015] FIG. 3 shows an example of a call-flow for creating a report for a sustainability assessment, according to an embodiment of the present disclosure.

[0016] FIG. 4A shows part of an example of a call-flow for an interview assessment, according to an embodiment of the present disclosure.

[0017] FIG. 4B shows part of an example of a call-flow for an interview assessment, according to an embodiment of the present disclosure.

[0018] FIG. 4C shows part of an example of a call-flow for an interview assessment, according to an embodiment of the present disclosure.

[0019] FIG. 5 shows an example of a call-flow for restarting an assessment, according to an embodiment of the present disclosure.

[0020] FIG. 6 shows an example of a user display for selecting a sustainability topic, according to an embodiment of the present disclosure.

[0021] FIG. 7 shows an example of a user display for uploading documents for a sustainability assessment, according to an embodiment of the present disclosure.

[0022] FIG. 8 shows an example of a user display for an interview assessment, according to an embodiment of the present disclosure.

[0023] FIG. 9 shows an example of a user display for a report from a sustainability assessment, according to an embodiment of the present disclosure.

[0024] FIG. 10 shows an example of a user display for a report from a sustainability assessment, according to an embodiment of the present disclosure.

[0025] FIG. 11 shows an example of a user display for selecting sustainability factors for comparison, according to an embodiment of the present disclosure.

[0026] FIG. 12 shows an example of a user display for recommendations for best practices, according to an embodiment of the present disclosure.

[0027] FIG. 13 shows a schematic of an apparatus for performing a sustainability assessment according to embodiments of the present disclosure.

[0028] FIG. 14 shows a schematic of an embodiment of an electronic device that may implement at least part of the methods and features of the present disclosure.

[0029] It will be noted that throughout the appended drawings, like features are identified by like reference numerals.DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure are generally directed towards providing Al tools for assessing practices in view of best practices for sustainability. Sustainability can include, but is not limited to, aspects of environmental impact, human wellbeing, resource consumption and generation, justice and equity, organizational governance, and societal impact. Sustainability can further encompass practices that contribute to the perpetuity of an organization, group, operation, activity, or society. Embodiments can include generative Al tools, especially those based on LLMs such as ChatGPT developed by OpenAI. In embodiments, these generative Al tools may be integrated with other elements to provide retrieval augmented generation (RAG). In this way, the generative Al tools can be informed by best practices provided by subject-matter experts (SMEs), documents detailing the practices of an organization, and information directly provided by a user.

[0031] Embodiments can be implemented by a computer device, which can include one or more processors coupled with tangible, non-transitory memory. The computer device can, for example, be a personal computer, a laptop, a tablet, or a smartphone. The computer device can further include or can be communicatively coupled with one or more databases. The computer device can have software that is stored on the memory and that has instructions to implement embodiments of the present disclosure. The software can include an application programming interface (API) that can push messaging events and related meta-data. The API can be configured to operate according to a representational state transfer (REST) architecture and can operate dynamically in response to real-time analysis of a sustainability assessment. A client function can be included to provide access to the API by a user. The software can include a generative Al tool or LLM, such as an OpenAI ChatGPT tool. In some embodiments, the software can utilize a plurality of different generative Al tools or LLMs for distinct tasks, including for response generation and subsequent response verification. Instructional prompts for the generative Al tool can be pre-determined to guide analysis of practices and provide recommendations for best practices. The software can further include an email service to provide notifications to a user and an master data managementsystem (MDMS), such as Directus, for administering data from the one or more databases. The one or more databases may be located at a mass storage of the computer device or, for example, at a remote server, such as at cloud computing infrastructure. The databases can include a vector database such as a Qdrant database, a remote database such as a Google Sheets database, and a file storage system such as an Amazon S3 cloud storage. The computer device can further be coupled to a user interface, which can include a display such as a monitor or screen, a keyboard, a printer, speakers, a microphone, a mouse, and / or other input and output devices.

[0032] Embodiments can include receiving input data from a user, who may, for example, be a manager of an organization or another party who is interested in assessing the practices of the organization. The input data can include selections, uploads of documents, responses to questions, follow-up questions, and confirmations of findings.

[0033] In some embodiments, an assessment can be provided based on documents defining the practices of an organization. In these embodiments, one or more documents containing text that defines the practices of an organization can be uploaded by a user through the user interface and subsequently received by the API. The API can then coordinate activities to provide a sustainability assessment. The documents can, for example, be computer files such as portable document format files (PDFs) and may, for example, be a report generated by the organization. The documents can be validated to ensure that they provide sufficient text for the assessment. The documents can be compiled into a vector collection. Vector embeddings for each document can be generated using the generative Al tool to represent the text thereof. This can include decomposing each document into sequential excerpts of text, to which the vector embeddings may correspond. The vectors embeddings can be indexed in a vector database, such as a Qdrant vector database, as a collection.

[0034] To assess the documents, they can be evaluated in consideration of one or more aspects of sustainability. These aspects may pose a question on the practices of an organization or define a best practice. For example, a question could be “What percentage of senior management are female?”. In some embodiments, the documents can be compared against text on best practices for sustainability. The best practices can be received from a subject-matter expert (SME) prior to anassessment and stored at one of the one or more databases. Vector embeddings for the best practices text can be extracted, and then the collection of vector embeddings corresponding to the documents can be searched for similarity to the vector embeddings of the best practices. Similarity can be assessed using a metric based on a dot product, cosine similarity, Euclidean distance, and / or a Manhattan distance. The search can be performed concurrently across multiple documents of the collection to find, extract, summarize, and analyze the most relevant document excerpts. This can provide indications of which best practices are being performed and evidence thereof (through document references). Alternatively, the aspects of sustainability may pose questions, in which case the collection of vector embeddings corresponding to the documents can be searched for similarity to vector embeddings of the questions. The results can then be used by the generative Al tool to compose an answer to each question to define the performance of the organization, while providing references to support in the documents.

[0035] In some embodiments, the answers produced by the analysis of documents can be compared against metrics for key performance indicators (KPIs). This can be done to determine a competitivity for the organization’s practices against other competitors. The comparison can include filtering and processing the answers, aggregating data therefrom, and aggregating data from competitors. For example, medians, means, minima, maxima, scores, and numerical values derived from other formulae can be generated from the answers and results. These can be stored at one of the one or more databases for further use or can be provided to the user.

[0036] In some embodiments, a chat or interview may be initiated with a user in regard to the results or answers produced by the analysis of the documents. The chat can use the generative Al tool to create questions for the user and to create responses to messages from the user. The generative Al tool can use the results as input in creating its own messages to send to the user or to shorten and summarize the results for provision to user. For example, the generative Al tool can create a message to request validation by the user of an answer or result that has been suggested by the generative Al tool. In some embodiments, the user can invoke an Al-based accuracy verification feature to assess the suggested answer or result, as described further below. In this case, the user can be presented with options to accept, reject, or edit the answer. Messages to the user can include references to the document, such as a page number, from which the results werederived. In another example, the messages can provide a recommendation for the user based on best practices for sustainability and the practices defined in the documents. Chat messages can be provided dynamically and conversationally to the user.

[0037] In some embodiments, an Al-based accuracy verification feature can be implemented to assess the validity and accuracy of responses generated by the generative Al tool. The accuracy verification feature provides a "second opinion" or "double check" mechanism to provide users with a convenient means of assessing the accuracy of a generated response, mitigating potential inaccuracies, biases, or hallucinations inherent in LLM operations. Utilizing the same LLM to verify its own response may be ineffective, as repeating the operation with the same conversational context and inputs may result in the reproduction of original errors due to the same biases and parametric memory, or may cause the LLM to arbitrarily change its response. Therefore, the accuracy verification feature utilizes an alternate generative Al tool (or an alternate LLM) to perform the accuracy assessment.

[0038] The accuracy verification feature can comprise several integrated components, including an LLM selection component and one or more LLM-based software agents configured with instructional prompts.

[0039] The LLM selection component can be configured to select the alternate LLM, ensuring it is distinct from the LLM used to generate the original response. In embodiments, the LLM selection component can dynamically fall back to other alternate LLMs in the event of errors, latency, or unavailability of the initially selected alternate LLM.

[0040] In operation, when verification of an original Al response is initiated, the LLM-based software agents can cooperate to gather and reconstitute the context of the original response from the conversational history. This reconstituted context can comprises several elements, including: the original user request and inferred intent; the external data provided to the original LLM during the initial operation; and the original response itself. The external data can include, for example, relevant search results from a knowledge base, such as document fragments retrieved from thevector database, and relevant web search results, retrieved from a search API or web search oriented LLM.

[0041] The selected alternate LLM can be then utilized to perform a structured accuracy assessment, using the reconstituted context as input. In some embodiments, this can be executed as a "one shot" operation guided by the instructional prompts. These instructional prompts can define parameters for the operation, which can include: temporal rules, for dealing with potential issues relating to interpretations of time values within the context; requirements for the use of evidence (derived from the external data) when performing the assessment; classification guidelines for determining an assessment result; and stylistic guidelines for generating an assessment response.

[0042] The output of the structured accuracy assessment can be structured data (structured output) designed to facilitate straightforward review by the user. The structured output can include an assessment result classification, for example, "Accurate", "Incomplete", or "Inaccurate".

[0043] When the assessment result is "Incomplete", the structured output can include details regarding any missing information.

[0044] When the assessment result is "Inaccurate", the structured output can include identification of the inaccurate information, detailing specific facts or information in the original response that are inaccurate, which may include citing parts of the original response verbatim. The output can further include reasons for the inaccuracies, such as information that contradicts other reputable sources (e.g., the provided external data), information that is out-of-date, or apparent "hallucination" by the original LLM. Furthermore, the alternate LLM can generate a new, corrected response that seeks to rectify the inaccuracies of the original response.

[0045] The structured output can further include a confidence score, reflecting a determination of likelihood by the alternate LLM that the assessment results themselves are accurate.

[0046] The assessment results can be stored in one of the one or more databases or the file storage system for future reference by the user. The user interface can be configured to display thestructured accuracy assessment results. In some embodiments, the user interface can also display identifying information, such as the vendors and names, of the two different LLM models used for the original response generation and the accuracy assessment response, respectively.

[0047] In some embodiments, multiple documents can be analyzed across multiple factors to compare sustainability practices across multiple organizations. In these embodiments, the name of the organization to which a particular document pertains can be extracted. Each document may be decomposed into excerpts, and vector embeddings can be extracted, as described above. The documents can be concurrently searched for data relating to multiple factors of sustainability. The search results may be validated by questions created by the generative Al tool and provided to a user. In embodiments, a report, or performance report, comparing the factors across all the documents can be generated.

[0048] In some embodiments, an assessment of sustainability can be provided based on descriptions of practices that are received from a user during an interview. In these embodiments, the interview may be conducted dynamically in real-time, with automatic changes in the direction of the interview made in response to messages from the user. The interview can further initiate side-bar chats to address topics that arise during the interview. A side-bar chat can, for example, be a window that opens separate from one for the interview on a user display. Questions for a user can be created by the generative Al tool. The questions can, for example, ask how a user addresses a particular aspect of sustainability and the user’s practices towards that aspect. An example of a question could include: “Has the leadership of the organization made a formal written commitment to driving diversity, equity, and inclusion across the organization (e.g., in your Annual Report, website or other publication)?”. A question could further ask a user to review a document or an excerpt of text. The generative Al tool can be used to assess if the response from a user answers a question appropriately. In the interview, the generative Al tool can be used to respond to questions posed by the user and assess whether a user intends to move the conversation off-topic or request assistance. These possibilities can also initiate side-bar chats. The generative Al tool can further be used to provide context to a user or references to supporting information in documents. For example, further instructions, rationale, or examples can be provided to the user. In some embodiments, an offer for a tip may be provided to the user to assist in providing an answer. Inthese embodiments, tips can be loaded from documents generated by a SME and modified for the conversation by the generative Al tool. The generative Al tool can select a relevant tip for the interview by comparing vector embeddings of the conversation and of the tips. As the interview progresses, messages can be recorded and stored for analysis. Analysis can be done to evaluate KPI metrics or generate a report summarizing the practices of the user towards sustainability.

[0049] In embodiments, analysis or results of a sustainability assessment can be automatically organized and presented as a report. The report can be automatically tailored to the needs of a user. The report may be compiled by the generative Al tool using pre-determined prompts. The generative Al tool can compile the results of an interview or assessment of documents by analysing and summarising the results. This information can be incorporated into different report sections that are relevant to the user. The generative Al tool can further create a checklist of recommendations for best practices that can be incorporated into the report.

[0050] In embodiments, organizations can be automatically classified into leadership categories in accordance with the results of an assessment. Organizations can, for example, be classified as leaders, followers, or laggards. In embodiments, the generative Al tool can use pre-determined instructional prompts to classify an organization according to each factor of assessment. Classification criteria can be defined by an SME, as further described below, or can be suggested by the generative Al tool. The generative Al tool can further create an explanation for the classification and provide references to assessment results from documents or an interview. A prompt provided to the generative Al tool can be, as an example: “As briefly and succinctly as possible, summarize the company's specific activities or actions (or lack thereof) that resulted in the classification that you assigned.”.

[0051] In embodiments, user inputs can include a selection of a type of task to be performed, such as a document assessment, document chat, document summarization, competitive analysis, or interview, and a selection of a form or area of sustainability to be assessed. These selections can be requested from the user to initiate an assessment.

[0052] In embodiments, best practices can be received from a SME. The SME can further define what information is needed from a user in order to form an assessment with the best practices. Embodiments can include data authoring software to enable a SME to populate information collection data schema for collecting data from users on their practices. The information collection data schema can include structures for providing ‘open-ended’ natural language questions on best practices, natural language validation expressions, sets of recommended activities for best practices, dependencies between data elements and conditional logic, and natural language criteria for assessing whether a given recommended activity is being practiced. The information collection data schema can thus provide an architecture for collecting valid and well-formed data.

[0053] Embodiments can further include data authoring software to enable a SME to populate report generation data schema for providing results of an assessment to a user. The report generation data schema can include structures for generating report sections with conditional transformation instructions that reference gathered information, recommendations with naturallanguage conditions and references to gathered information, and recommendations that are specific to particular data elements and reporting processes.

[0054] Embodiments can further include data authoring software to enable a SME to populate comparison factor data schema for defining factors to compare and classify organizations following their analysis. The comparison factor data schema can include structures for factor titles, factor descriptions, factor metadata, classification criteria, and numerical classification scales.

[0055] In embodiments, the user interface can include a software component for communicating information to a user. This can include communicating suggestions for performed and under- performed practices, questions and answers, interview messages, excerpts from documents with supporting references, estimates of time required for an assessment, interview progress, sidebar conversations, and analysis results and reports. This can further include communicating the results of the structured accuracy assessment. Communicating this information to the user can include displaying the information on a screen or providing files with the information for download. In some embodiments, the information, such as a report, can be emailed to the user by an email service.

[0056] Embodiments of the present disclosure can automate sustainability assessments by integrating generative Al tools with structures for collecting and processing data on practices for sustainability. The automation can reduce the processing time for an assessment and remove bias and subjectivity from the assessment. Through the evaluation of user-provided documents or interview responses, assessments can use well-formed, relevant data. Embodiments further enable real-time responses and analysis that cannot be achieved through manual assessments or without the integrated structures.

[0057] The present disclosure sets forth various embodiments via the use of block diagrams, flowcharts, and examples. Insofar as such block diagrams, flowcharts, and examples contain one or more functions and / or operations, it will be understood by a person skilled in the art that each function and / or operation within such block diagrams, flowcharts, and examples can be implemented, individually or collectively, by a wide range of hardware, software, firmware, or combination thereof. As used herein, the term “about” should be read as including variation from the nominal value, for example, a + / -10% variation from the nominal value. It is to be understood that such a variation is always included in a given value provided herein, whether or not it is specifically referred to.

[0058] FIG. 1 shows an example of a call-flow of a method for a competitive analysis for sustainability using documents, in accordance with an embodiment of the present disclosure. The method can be implemented by a client function 101 serving a user, a REST API 102, a database 104, a vector database 105, an LLM 106, a file storage system 107, and an email service 108. The documents to be analyzed can be PDF files and can be stored at the file storage system 107. At action 109, the client 101 messages the REST API 102 to create a batch task for the competitivity analysis. At action 110, questions for the analysis are read from the database 104 and provided to the REST API 102. Each question can include a prompt for the Open Al generative Al tool 106. For example, a question could be:If this company has targets to reduce its greenhouse gas or carbon emissions, list the targets, their due dates, and what they cover, in bullet points, without repeating back elements of the question. Examples of the expected responses to this query include: ‘30% reduction inscope 1 emissions by 2025 and 50% reduction in scopes 1, 2, and 3 emissions by 2030’, ‘20% reduction in scopes 1 and 2 emissions by 2025’, ‘30% reduction in business travel emissions by 2026’ . If no numerical information is found on this, and there is only mention of targets existing in the text, return a short sentence stating what the company says about its future targets. If no information at all is found on future targets, return ‘No mention of future targets found’.At action 111, a task record can be created at the database 103. At action 112, mappings for the PDF files are retrieved by the REST API 102 from the database 103. Mappings may store metadata for the PDF files, such as their respective filenames and identifiers. Actions 113 to 118 are performed for each PDF and for each question read from the database 104. For each PDF, the vector database 105 checks to see if a corresponding collection exists. If a collection does not exist: a collection is created, at action 114; the respective PDF file is retrieved from the file storage system 107, at action 115; and, for each page or portion of text in the PDF file, a respective vector embedding is created in the collection, at action 116. If a collection does exist, the REST API 102 is notified by the vector database 105. At action 117, the collection is searched for vectors embeddings similar to those of the respective question. Once this data for the respective PDF is retrieved, it is provided to the LLM 106, at action 118, and an answer to the respective question is created for the REST API 102. At action 119, an email is sent from the email service 108 to notify the user that the analysis is complete. At action 120, the status of the task is updated to reflect its completion.

[0059] FIG. 2 shows an example of a call-flow of a method for a chat assessment of multiple documents, in accordance with an embodiment of the present disclosure. The assessment can be performed to answer a question provided by a user through a generative Al tool. The method can be implemented by a client function 101 serving the user, a server or API 102, a vector database 105, an LLM 106, and a file storage system 107. The documents for the chat assessment can be PDF files and can be stored at the file storage system 107. At action 201, the client function 101 provides the API 102 with a user message initiating the assessment of the PDFs. At action 202, the API 102 retrieves the PDF files from the file storage system 107. Actions 203 to 219 are performed for each PDF. At action 203, the API 102 checks whether a session state exists at thefile storage system 107 for the respective PDF. The session state can be a set of information that is used to answer the question, such a record of messages between the user and the generative Al tool. If a session state exists: the API 102 is notified, at action 204; the vector database 105 is then searched for data from the PDF that is similar to the question, at action 205, by obtaining vector embeddings for the question using the LLM 106, at action 206, and performing a similarity search between the vector embeddings for the question and the data from the PDF, at action 207; a response to the question is created using the LLM 106, at action 208; and the session state is updated, at action 209. If a session state does not exist: the session state is initialized, at action 210; the PDF file is prepared, at action 211, by retrieving vector embeddings from the PDF text using the LLM 106, at action 212, and creating a collection at the vector database 105 with the vector embeddings, at action 213; the vector database 105 is then searched for data from the PDF that is similar to the question, at action 214, by obtaining vector embeddings for the question using the LLM 106, at action 215, and by performing a similarity search between the vector embeddings for the question and the data from the PDF, at action 216; a response to the question is created using the LLM 106, at action 217; and the session state is updated, at action 218. At action 219, a chat document message is created for each document according to the response to the question. At action 220, the chat messages are provided to the user.

[0060] FIG. 3 shows an example of a call-flow of a method for creating a report based on the assessment of multiple documents, in accordance with an embodiment of the present disclosure. The method can be implemented by a client function 101 serving a user, a REST API 102, a database 104, a vector database 105, an LLM 106, a file storage system 107, and a MDMS 301. The documents for the chat assessment can be PDF files. At action 302, the REST API 102 receives a message from the client function 101 to create a report. At action 303, the database 104 is requested to validate data for the report, and, at action 304, file identifiers (IDs) for the PDF files for the report are checked at the database 104. If at least some of the file IDs are invalid, an error message is provided, at action 305, to the REST API 102 from the database 104 and subsequently to the client function 101. If all the file IDs are valid: the text of the PDF files is validated by the vector database 105, at action 306, and actions 307 to 317 may be performed. If the text of the PDF files is invalid, an error message is provided, at action 307, to the REST API 102 andsubsequently to the client function 101. If the text of the PDF files is valid: a sustainability topic is retrieved from the user by the MDMS 301, at action 308; a record for the report is created at the database 104, at action 309; each PDF file is retrieved from the user by the MDMS 301, at action 310; a collection for the report is created at the vector dataset 105, at action 311, by creating vector embeddings for excerpts extracted from each PDF file, at action 312; the collection is searched for data related to each of one or more questions to be answered for the report (e.g., as defined by a report definition schema provided by an SME), and answers to the questions with supporting references are created by the LLM 106, at action 313; each answer is shortened by the LLM 106, at action 314; each answer is checked by the LLM 106 to confirm that it answers the corresponding question, at action 315; the answers are saved as a JavaScript Object Notation (JSON) file at the file storage system 107, at action 316; and the status of the report at the database 104 is updated. At action 318, the client function 101 is notified of the creation of the report.

[0061] FIGs. 4A, 4B, and 4C show an example of a call-flow of a method for producing a report for a sustainability assessment based on an interview, in accordance with an embodiment of the present disclosure. The method can be implemented by a client function 101 serving a user, a REST API 102, a database 104, a vector database 105, a LLM 106, a file storage system 107, and a MDMS 301.

[0062] At action 401, a user messages, by the client function 101, the REST API 102 to initiate the production of the report. At action 402, the database 104 is checked to determine whether a report for the interview currently exists and the status thereof. For example, if a user has just initiated interaction with the REST API 102, a report may not exist, but if the user has previously interacted with the REST API 102, a report of previous messages may be available. If the report does not exist, an error message is sent to the client function 101, at action 403. If the report exists but its status indicates that the interview status has been terminated, an error message is sent to the client function 101, at action 404. If the report exists and the status indicates that the interview has an active status, the content of message sent by the user is validated by the REST API 102, at action 405, and actions 406 to 444 may be performed. If the content of the message is invalid, an error message is provided to the client function 101. If the content of the message is valid, actions 406 to 444 are performed. At action 406, a request for a state of the interview session is sent to thefile storage system 107. If the session state exists, it is retrieved. If the session state does not exist: answers, such as those provided in relation to the method of FIG. 3, are loaded as a JSON file from the file storage system 107, at action 407; the report is obtained from the database 104, at action 408; a PDF document containing sustainability tips is loaded from the file storage system 107, at action 409; a chat is initiated with the LLM 106 and a message is provided to the LLM that defines how it should respond to forthcoming messages from the user, at action 410; and the session state is saved at the file storage system 107, at action 411, and initialized, at action 412. The sustainability tips can be pre-defined recommendations for best practices and / or for answering questions and can be provided by an SME.

[0063] At action 413, the answers are loaded as a JSON file from the file storage system 107. At action 414, the report is obtained from the database 104. At action 415, the session state is obtained from the file storage system 107. At action 416, an offer for a tip is provided by the LLM 106 using a last message that was provided to the user and a response is obtained.

[0064] If the tip response does not indicate that the user does not need help: a corresponding tip is searched for, at action 417, using the vector database 105 and the tips loaded previously according to vector embeddings created for the tips; the tip is confirmed as a relevant match by the LLM 106, at action 418; the tip is made presentable or beautified by the LLM 106, at action 419, if it is confirmed; a new tip is created by the LLM 106, at action 420, if the tip is not confirmed as a relevant match; and the session state of the interview is saved at the file storage system 107, at action 421.

[0065] If the tip response does indicate that the user does not need help, the user is asked a question by the LLM 106 and an answer is received, at action 422. When the answer is over, actions 423 to 429 are performed. At action 423, if the question has dependencies, the conversation with the user is checked to determine if an expected answer has been received. This can include checking the similarity of the answer with the conversation. At action 424, the session state of the interview is saved at the file storage system 107. At action 425, the report is saved. At action 426, the answers are loaded as a JSON file from the file storage system 107. At action 427, the conversation with the user is summarized by the LLM 106. At action 428, each answer is validated with regard to itscorresponding question by the LLM 106. At action 429, the answers are saved as a JSON file at the file storage system 107. While the answer is not over, the session state is saved at the file storage system 107, at action 430. At action 431 the session state is obtained, and at action 432, the report is obtained.

[0066] If, when the report is returned from the database 104, the status of the report indicates terminated, actions 433 to 443 may be performed. At action 433 definitions for the generation of a report are obtained from the MDMS 301, such as according to a report generation data scheme provided by an SME. At action 434, the JSON file with the answers is loaded from the file storage system 107. At action 435, compatibility of the JSON file is checked by the REST API 102. This can include validating that the JSON file is well-formed and adheres to the expected report generation data scheme provided by an SME. If the JSON file is incompatible, or has a mismatch, the report is saved with a state of being in progress at the file storage system 107. If the JSON file is compatible, actions 437 to 443 may be performed. At action 437 the report is obtained from the file storage system 107. At action 438, the JSON file with the answers is loaded from the file storage system 107. At action 439, the definitions for the generation of the report are obtained from the MDMS 301. At action 440, a paragraph is created by the LLM 106 for each section of the report, such as in accordance with a report generation data scheme. If the definitions indicate that a section for all recommendations of the assessment should be created, that section is created in the report by the LLM 106, at action 441. If the definitions indicate that a section for a particular recommendation should be created, that section is created in the report by the LLM 106, at action 442. At action 443, the data for a finalized report is saved at the file storage system 107.

[0067] At action 444, the conversation messages are provided to the client function 101 for display to the user, and at action 445, a message is provided to the client function 101.

[0068] FIG. 5 shows an example of a call-flow of a method for restarting an interview for a sustainability assessment, in accordance with an embodiment of the present disclosure. The method can be implemented by a client function 101 serving a user, a REST API 102, a database 104, a database 104, an LLM 106, and a file storage system 107. At action 501, the client function 101 messages the REST API 102 to initiate the restart. At action 502, the database 104 is searchedto determine whether a report exists and the status thereof. If a report does not exist, an error message is sent to the client function 101, at action 503. If a report does exist and the status indicates that it is terminated: the validity of questions in the report are checked by the REST API 102, at action 504; a session state is obtained from the file storage system 107, at action 505; the report is obtained from the database 104, at action 506; the report status is updated to being in progress at the database 104, at action 507; the session state is saved at the file storage system 107, at action 508; answers from the assessment are loaded as a JSON file from the file storage system 107, at action 509; the report is obtained from the database 104, at action 510; the session state is obtained from the file storage system 107, at action 511; a new chat is initiated with the LLM 106 and a message is generated therefrom, at action 512; the session state is saved at the file storage system 107; and a first question for the user is selected and provided to the client function 101, at action 514. If a report does exist but the status indicates that it is active, an error message is provided, at action 515, to the client function 101 indicating that a current interview is not yet complete.

[0069] FIG. 6 shows an example of a display 600 for choosing a sustainability topic, in accordance with an embodiment of the present disclosure. The display 600 can include a direction to a user to select a sustainability topic for assessment and a plurality of sustainability topics 601 that can be selected. Each sustainability topic 601 can have a respective title and description.

[0070] FIG. 7 shows an example of a display 700 for uploading documents for a sustainability assessment, in accordance with an embodiment of the present disclosure. The display 700 can include instructions for the user to upload documents as well as buttons that enable the user to select 701 the documents and to upload 702 the selected documents.

[0071] FIG. 8 shows an example of a display 800 for communicating chat or interview messages to a user and receiving responses therefrom, in accordance with an embodiment of the present disclosure. The display 800 can include a record of sent messages 801 provided to a user, such as messages generated by a generative Al tool, and received messages 802 from the user. The display 800 can further include a messaging field 803 for messages being drafted by the user.

[0072] FIG. 9 shows an example of a display 900 for a report generated based on a sustainability assessment of documents, in accordance with an embodiment of the present disclosure. The display 900 can show a title for the report as well as sections 901 on different factors of sustainability. Each section 901 can include, for example, answers to questions from the assessment, measures of KPI metrics, and / or recommendations for best practices. The text of each section 901 can, for example, be a summary prepared by a generative Al tool of excerpts from the documents that were found by a similarity search of vector embeddings.

[0073] FIG. 10 shows an example of a display 1000 for a report based on an interview and documents, in accordance with an embodiment of the present disclosure. The display 1000 can include questions 1001 that were provided to a user in the interview or to analyze documents, responses 1002 to the questions 1001, supporting evidence 1003 from the documents or interview with references thereto, and recommended activities 1004.

[0074] FIG. 11 shows an example of a display 1100 for generating a report, in accordance with an embodiment of the present disclosure. The display 1100 can include instructions to a user and options 1101 for factors or points of comparison that can be included in the report. The options 1101 can, for example, be prompts to queries on different topics of sustainability. The factors can be grouped under different sustainability topics.

[0075] FIG. 12 shows an example of a display 1200 for recommendations based on a sustainability assessment, in accordance with an embodiment of the present disclosure. The display 1200 can include a summary of sustainable practices by a user, which can be created by a generative Al tool. The display 1200 can further include one or more checklists of best practices 1201 that are being practiced by the user or not being practiced. The display 1200 can further provide options 1202 for confirming the practice of best practices on a checklist.

[0076] Embodiments of the present disclosure can be implemented using electronics hardware, software, or a combination thereof. Some embodiments can be implemented by one or multiple computer processors executing program instructions stored in memory. Some embodiments can be implemented partially or fully in hardware, for example using one or more field programmablegate arrays (FPGAs) or application specific integrated circuits (ASICs) to rapidly perform processing operations.

[0077] FIG. 13 shows an apparatus 1300 for performing a sustainability assessment with Al tools, according to embodiments of the present disclosure. The apparatus 1300 can be located at a node 1310 of a network. The apparatus can include a network interface 1320 and processing electronics 1330. The processing electronics 1330 can include a computer processor executing program instructions stored in memory, or other electronics components such as digital circuitry, including for example FPGAs and ASICs. The network interface 1320 can include an optical communication interface or radio communication interface, such as a transmitter and receiver. The apparatus 1300 can include several functional components, each of which may be partially or fully implemented using the underlying network interface 1320 and processing electronics 1330. Examples of functional components may include modules for receiving 1340 user input, creating 1341 vector embeddings, searching 1342 vector embeddings, analyzing 1343 practices, and reporting 1344 recommendations.

[0078] FIG. 14 shows a schematic diagram of an electronic device 1400 that can perform any or all of the operations of the above methods and features explicitly or implicitly described herein, according to different embodiments of the present disclosure. For example, a computer equipped with network function can be configured as electronic device 1400. The electronic device 1400 can be used to implement the apparatus 1300 of FIG. 13, for example.

[0079] As shown, the electronic device 1400 can include a processor 1410, such as a Central Processing Unit (CPU) or specialized processors such as a Graphics Processing Unit (GPU) or other such processor unit, memory 1420, and a bi-directional bus 1430 to communicatively couple the components of electronic device 1400. Electronic device 1400 can also optionally include a network interface 1440, non-transitory mass storage 1450, an VO interface 1460, and a transceiver 1470. According to certain embodiments, any or all of the depicted elements can be utilized, or only a subset of the elements. Further, the electronic device 1400 can contain multiple instances of certain elements, such as multiple processors, memories, or transceivers. Also, elements of the hardware device can be directly coupled to other elements without the bi-directional bus 1430. 1Additionally or alternatively to a processor and memory, other electronics, such as integrated circuits, can be employed for performing the required logical operations.

[0080] The memory 1420 can include any type of tangible, non-transitory memory such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), any combination of such, or the like. The mass storage element 1450 can include any type of tangible, non-transitory storage device, such as a solid-state drive, hard disk drive, a magnetic disk drive, an optical disk drive, USB drive, or any computer program product configured to store data and machine executable program code. According to certain embodiments, the memory 1420 or mass storage 1450 can have recorded thereon statements and instructions executable by the processor 1410 for performing any of the aforementioned method operations described above.

[0081] Network interface 1440 can include at least one of a wired network interface and a wireless network interface. The network interface 1440 can include a wired network interface to connect to a communication network and can also include a radio access network interface for connecting to the communication network or other network elements over a radio link. The network interface 1440 enables the electronic device 1400 to communicate with remote entities such as those connected to the communication network. This can include enabling the electronic device to be communicatively coupled with one or more remote databases, such as a database 103, a file storage system 107, or a Qdrant vector database 105, as well as services provided through the network, such as an LLM 106.

[0082] The I / O interface 1460 can include elements for communicating with a user, such as a as a monitor or screen, a keyboard, a printer, speakers, a microphone, a mouse, and / or other input and output devices.

[0083] It will be appreciated that, although specific embodiments of the technology have been described herein for purposes of illustration, various modifications can be made without departing from the scope of the technology. The specification and drawings are, accordingly, to be regarded simply as an illustration of the invention as defined by the appended claims, and are contemplatedto cover any and all modifications, variations, combinations or equivalents that fall within the scope of the present invention. In particular, it is within the scope of the technology to provide a computer program product or program element, or a program storage or memory device such as a magnetic or optical wire, tape or disc, or the like, for storing signals readable by a machine, for controlling the operation of a computer according to the method of the technology and / or to structure some or all of its components in accordance with the system of the technology.

[0084] Acts associated with the method described herein can be implemented as coded instructions in a computer program product. In other words, the computer program product is a computer- readable medium upon which software code is recorded to execute the method when the computer program product is loaded into memory and executed on the microprocessor of the wireless communication device.

[0085] Further, each operation of the method can be executed on any computing device, such as a personal computer, server, PDA, or the like and pursuant to one or more, or a part of one or more, program elements, modules or objects generated from any programming language, such as C++, Java, or the like. In addition, each operation, or a file or object or the like implementing each said operation, can be executed by special purpose hardware or a circuit module designed for that purpose.

[0086] Through the descriptions of the preceding embodiments, the present invention can be implemented by using hardware only or by using software and a necessary universal hardware platform. Based on such understandings, the technical solution of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile or non-transitory storage medium, which can be a compact disk read-only memory (CD-ROM), USB flash disk, or a removable hard disk. The software product can include a number of instructions that enable a computer device (personal computer, server, or network device) to execute the methods provided in the embodiments of the present disclosure. For example, such an execution can correspond to a simulation of the logical operations as described herein. The software product can additionally or alternatively include number of instructions that enable acomputer device to execute operations for configuring or programming a digital logic apparatus in accordance with embodiments of the present disclosure.

[0087] The word “a” or “an” when used in conjunction with the term “comprising” or “including” in the claims and / or the specification can mean “one”, but it is also consistent with the meaning of “one or more”, “at least one”, and “one or more than one” unless the content clearly dictates otherwise. Similarly, the word “another” can mean at least a second or more unless the content clearly dictates otherwise. The phrase "at least one" means one or more, and "a plurality of means two or more. In addition, "and / or" describes an association relationship of associated objects, and indicates that there can be three relationships. For example, A and / or B can indicate cases including “only A”, “both A and B”, and “only B”, where A and B can be singular or plural. The character " / " generally indicates that the associated objects are in an OR relationship. "At least one of the following items" or a similar expression thereof refers to any combination of these items, including any combination of a single item or a plurality of items. For example, “at least one of a, b, or c” can represent “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, or “a, b and c”, where a, b, and c can be a single or multiple form.

[0088] The terms “coupled”, “coupling” or “connected” as used herein can have several different meanings depending on the context in which these terms are used. For example, as used herein, the terms coupled, coupling, or connected can indicate that two elements or devices are directly connected to one another or connected to one another through one or more intermediate elements or devices via an electronic element depending on the particular context. The term “and / or” herein when used in association with a list of items means any one or more of the items comprising that list.

[0089] Although a combination of features is shown in the illustrated embodiments, not all of them need to be combined to realize the benefits of various embodiments of this disclosure. In other words, a system or method designed according to an embodiment of this disclosure will not necessarily include all features shown in any one of the Figures or all portions schematically shown in the Figures. Moreover, selected features of one example embodiment may be combined with selected features of other example embodiments.

[0090] Although the present invention has been described with reference to specific features and embodiments thereof, it is evident that various modifications and combinations can be made thereto without departing from the invention. The specification and drawings are, accordingly, to be regarded simply as an illustration of the invention as defined by the appended claims, and are contemplated to cover any and all modifications, variations, combinations or equivalents that fall within the scope of the present invention.

Claims

WHAT IS CLAIMED IS1. A method comprising, at a computer device including a processor coupled to tangible, non-transitory processor- readable memory, the computer device further coupled to a user interface and a plurality of databases: receiving, by the user interface: one or more first user inputs providing a corresponding one or more document files each defining respective text representing a plurality of sustainability practices, and one or more second user inputs defining an assessment task; retrieving, from one database of the plurality of databases, one or more sustainability aspects; performing, for each document file of the one or more document files: decomposing the text of the respective document file into a respective plurality of excerpts; calculating, by a generative artificial intelligence (Al) model for each one of the respective plurality of excerpts, a respective vector embedding; and storing, at a further one of the plurality of databases, each vector embedding calculated for the respective document file; searching the further one database in accordance with one of the one or more sustainability aspects to determine one or more relevant excerpts from among the respective plurality of excerpts of each document file, the respective vector embedding of each relevant excerpt having a certain similarity to a vector embedding of the sustainability aspect and completing, by the generative Al model, the assessment task in accordance with the one or more relevant excerpts.

2. The method of claim 1 wherein: receiving, by the user interface, the one or more second user inputs defining the assessment task includes: receiving, by the user interface, a question; completing, by the generative Al model, the assessment task in accordance with the one or more relevant excerpts, includes: creating, by the generative Al model and in accordance with the one or more relevant excerpts, an answer to the question; and the method further comprises, at the computer device: providing, by the user interface, the answer.

3. The method of claim 1 wherein: receiving, by the user interface, the one or more second user inputs defining the assessment task includes: receiving, by the user interface, a summary request; completing, by the generative Al model, the assessment task in accordance with the one or more relevant excerpts, includes: creating, by the generative Al model, a practice summary in accordance with the one sustainability aspect and the one or more relevant excerpts; and the method further comprises, at the computer device: providing, by the user interface, the practice summary.

4. The method of claim 3 wherein: receiving, by the user interface, the one or more second user inputs defining the assessment task further includes: receiving, by the user interface, a competitive analysis request; andcompleting, by the generative Al model, the assessment task in accordance with the one or more relevant excerpts, further includes: determining, in accordance with the practice summary, one or more metrics for a corresponding one or more key performance indicators.

5. The method of claim 1 wherein: receiving, by the user interface, the one or more second user inputs defining the assessment task includes: receiving, by the user interface, a document analysis request; completing, by the generative Al model, the assessment task in accordance with the one or more relevant excerpts, includes: creating, by the generative Al model, an analysis of the one or more document files in accordance with the one sustainability aspect and the one or more relevant excerpts.

6. The method of claim 1 wherein: each sustainability aspect is a respective sustainability question; and the method further comprises, at the computer device: creating, by the generative Al model, a performance answer in accordance with the one sustainability aspect and the one or more relevant excerpts.

7. The method of claim 6 further comprising, at the computer device: creating a performance report including the performance answer.

8. A method comprising, at a computer device including a processor coupled to tangible, non-transitory processor- readable memory, the computer device further coupled to a user interface and a plurality of databases:creating, by a generative artificial intelligence (Al) model, a question about one or more aspects of sustainability; providing, by the user interface, the question; receiving, by the user interface, a user answer in response to the question; retrieving, from one of the plurality of databases, one or more sustainability best practices; creating, by the generative Al model, one or more sustainability recommendations in accordance with the user answer and the one or more sustainability best practices; and providing, by the user interface, the one or more sustainability recommendations.

9. The method of claim 8 wherein: providing, by the user interface, the one or more sustainability recommendations includes: generating a sustainability report including the one or more sustainability recommendations; and providing, by the user interface, the sustainability report.

10. The method of claim 8 wherein: the method further comprises, at the computer device: retrieving, from a further one of the plurality of databases, a set of sustainability practices; creating, by the generative Al model, a suggested answer in accordance with the set of sustainability practices; and providing, by the user interface, the suggested answer; and the user answer includes a validation for the suggested answer.

11. The method of claim 8 further comprising:retrieving, from a further one of the plurality of databases, a set of answer tips; performing, for each previous message of a set of previous messages: decomposing the respective previous message into a respective plurality of excerpts; and calculating, by a generative artificial intelligence (Al) model for each one of the respective plurality of excerpts, a respective vector embedding; and searching the set of tips in accordance with the set of previous messages to determine one or more relevant tips from among the set of tips, the one or more relevant tips each having a respective vector embedding having a certain similarity to the respective vector embedding of at least one previous message of the set of previous messages.

Images