A method and a system for optimizing interaction with a large pre-trained model

Abstract

A system and a method for optimizing interaction with a large pre-trained Model (LPTM) including the actions of obtaining input data from an input system, storing the input data in at least one database, creating a data window, the data window including a predefined amount of data recently acquired by the at least one database, and providing the data window to the large pre-trained model (LPTM).

Description

[0001] A Method and a System for Optimizing Interaction with a Large Pre- Trained Model

[0002] FIELD

[0003] The method and apparatus disclosed herein are related to the field of artificial intelligence (Al), and more particularly but not exclusively to optimizing interaction with a large pre-trained Al model (L.P.T.M) such as a large language model (LLM), and more particularly but not exclusively to managing the interaction memory of a large language model (LLM), or a similar large pre-trained Al model.

[0004] BACKGROUND

[0005] Artificial intelligence (Al) and particularly large pre-trained models (LPTM) as well as large language models (LLM) are expensive, requiring large storage systems and much processing power. LPTMs and LLMs in particular support interaction with a user (sometimes referred as an Al agent), where the user may interact with the LPTM repeatedly in a conversation mode. However, the LPTM requires that in every' instance of the conversation, all the history of the interaction is fed into the LPTM. The repeated feeding of the entire history of the conversation consumes increased amount of time and processing power each time the conversation is repeated. This makes the long interaction both slower and more expensive. Feeding streaming content such as audio and particularly video is therefore even more expensive. There is therefore a need for a method and a system that may overcome these deficiencies.

[0006] SUMMARY OF THE INVENTION

[0007] According to one exemplary embodiment, there is provided a computer- implemented method for optimizing interaction with a large pre-trained Al Model (LPTM), the method including: obtaining input data from an input system, storing the input data in at least one database, creating a data window, the data window including a predefined amount of data recently acquired by the at least one database, and providing the data window' to the large pre-trained Al model (LPTM). According to another exemplary embodiment, the computer-implemented method may also include: if the LPTM responds with a request for required data, composing a search query in the at least one database for the required data, adding the data acquired from the database responsive to the query to the data window to form an extended data window, and providing the extended data window to the LPTM.

[0008] According to yet another exemplary' embodiment, the computer-implemented method may also include: if the LPTM responds with LPTM provided content, adding the LPTM provided content to the at least one database, and communicating the LPTM provided content to the user.

[0009] According to still another exemplary' embodiment, the computer-implemented method may also include: the user data includes at least one of audible and visual data, the predefined amount of data recently acquired includes at least one of: a predefined number of seconds preceding a time-stamp associated with the data recently acquired, and a measure in seconds of the amount of data recently acquired following the timestamp.

[0010] Further according to another exemplary embodiment, the computer- implemented method may also include: the user data includes at least one of audible and visual data, the storing the user data in the at least one database includes storing at least one embedding data associated with the at least one of audible and visual data, and a search query includes the at least one embedding data.

[0011] Unless otherwise defined, all technical and scientific tenns used herein have the same meaning as commonly understood by one of ordinary skill in the relevant art. The materials, methods, and examples provided herein are illustrative only and not intended to be limiting. Except to the extent necessary or inherent in the processes themselves, no particular order to steps or stages of methods and processes described in this disclosure, including the figures, is intended or implied. In many cases the order of process steps may vary without changing the purpose or effect of the methods described. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Various embodiments are described herein, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments only, and are presented in order to provide what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the embodiment. In this regard, no attempt is made to show structural details of the embodiments in more detail than is necessary for a fundamental understanding of the subject matter, the description taken with the drawings making apparent to those skilled in the art how the several forms and structures may be embodied in practice.

[0013] In the drawings:

[0014] Fig. 1 is a simplified block diagram of an LPTM optimization system;

[0015] Fig. 2 is a flow chart of the main process of the memory optimization system;

[0016] Fig. 3 is a flow chart of data batch preparation process of memory optimization system; and

[0017] Fig. 4 is a flow chart of the LPTM interaction module of the memory optimization system.

[0018] DESCRIPTION OF THE EMBODIMENTS

[0019] The present embodiments comprise a method, one or more devices, and one or more software programs for optimizing interaction with an artificial intelligence (Al) system or software, and particularly (but not exclusively), with a large pre-trained model (LPTM).

[0020] The principles and operation of the system, a method, and / or a computer program for optimizing interaction with an LPTM according to the several exemplary embodiments may be better understood with reference to the following drawings and accompanying description. Before explaining at least one embodiment in detail, it is to be understood that the embodiments are not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. Other embodiments may be practiced or carried out in various ways. Also, it is to be understood that the phraseology and tenninology employed herein is for the purpose of description and should not be regarded as limiting.

[0021] In this document, an element of a drawing that is not described within the scope of the drawing and is labeled with a numeral that has been described in a previous drawing has the same use and description as in the previous drawings. Similarly, an element that is identified in the text by a numeral that does not appear in the drawing described by the text, has the same use and description as in the previous drawings where it was described.

[0022] The drawings in this document may not be of any scale. Different Figures may use different scales and different scales can be used even within the same drawing, for example different scales for different views of the same object or different scales for the two adjacent objects.

[0023] The phrases ‘at least one’, ‘one or more’ and ‘and / or’, etc. are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions ‘at least one of A, B and C. ‘at least one of A, B, or C’. ‘one or more of A, B, and C’, ‘one or more of A, B, or C’, and ‘A, B, and / or C may mean 'A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together’ .

[0024] The terms ‘a’ or ‘an entity’ may refer to one or more of that entity. As such, the terms ‘a’ (or ‘an’), ‘one or more1and ‘at least one1can be used interchangeably herein. It is also to be noted that the terms ‘comprising’, ‘including’, and ‘having’ can be used interchangeably.

[0025] Reference throughout this specification to ‘'one embodiment,” '‘an embodiment,” or similar language means that a particular feature, structure, or characteristic that is described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases “in one embodiment,” “in an embodiment, and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0026] The term ‘plurality’, as used herein, is defined as two or more than two. The term ‘another’, as used herein, is defined as at least a second or more. The term ‘coupled’, as used herein, is defined as connected, although not necessarily directly, and not necessarily mechanically.

[0027] In this document, the term ‘computing device’ may refer to any type of computing machine, including but not limited to, a computer, a portable computer, a laptop computer, a tablet computer, a mobile communication device, a network server, a cloud computer, etc., as well as any combination thereof. Such computing device or computing machine may include any ty pe or combination of devices, including, but not limited to, a processor or a processing device, a memory device, a storage device, a user interface device, and / or a communication device.

[0028] The terms ‘execute’, ‘perform’, ‘compute’, ‘calculate’, ‘process', etc. may refer to a processor of a computational device executing a software program code embodied on a non-transitory computer readable medium to achieve a result such as described after any of the terms ‘execute’, ‘perform’, ‘compute’, ‘calculate’, ‘process’, etc.

[0029] The term ‘client computing device’, or ‘client device’, ‘user device’ may refer to any type of computing device that is directly used, or operated, by a user. Such a device may include a user interface that may be used by a user directly, including means for user input and / or user output. Such a device may be communicatively coupled to another computing devices such as a network server via a communication network.

[0030] Means for user input may include a keyboard, a pointing devices such as a mouse, a microphone, a camera, a touch-sensitive plate, or display, means for user gesture control, means for haptic user control, etc. Other means that may be considered as ‘user input’ may include various sensors such as inertial measuring units, heartbeat monitors, blood oxygen monitors, temperature monitors, etc.

[0031] It is appreciated that the term ‘user’ above may refer to a human user. However, the term ‘user’ may also refer to a machine, such as any type of computerized device and / or a software package. Particularly, the term ‘user’ may also refer to an Al system interacting with another Al system (LPTM).

[0032] Means for user output (namely, output to a user) may include a display, and / or any other means for providing visual information, a speaker, or an earphone, and / or any other means for providing audible information, means for providing tactile and / or haptic information, etc. Means for ‘user output’ where the ‘user’ is a machine (system) may be any means of computer communication (e.g., a communication network).

[0033] The term 'mobile communication device’ may refer to devices such as a tablet, a mobile telephone, a smartphone, etc.

[0034] The term ‘network server’ or ‘server’ may refer to any type of ‘computing device’ that is communicatively coupled to a communication network and may include a cloud computer, etc.

[0035] The term 'communication network’ or ‘network’ may refer to any type or technology for digital communication including, but not limited to, the Internet, WAN, LAN, MAN, PSDN, etc. Any of the abovementioned technologies may be wired or wireless, for example, Wireless WAN such as WiMAX, WLAN (Wi-Fi), WPAN (Bluetooth), etc. Wireless networking technology may also include PLMN, and / or any type of cellular network. The term ‘communication network’ or ‘network’ may refer to any combination of communication technologies, and to any combination of physical networks. The term ‘communication network’ or ‘network’ may refer to any number of interconnected communication networks that may be operated by one or many network operators.

[0036] The term ‘communication’ may refer to the use of any communication network, or means of communication, by a user (person, human) to communicate content to another user. Such communication may be direct like in a telephone call, or indirect (or store and forward), such as in messaging. Messaging can be half-duplex, for example, when the message is completed, stored, forwarded to the recipient, and then consumed by the recipient in whole before responding to the sender. Messaging can be full-duplex, for example, when the message may be forwarded to the recipient before it is completed and the recipient may respond to the sender before the message ends. The terms ‘information’, ‘content’, and ‘medium’ (or ‘media’) may refer to any type of data generated by a human (e.g., using an input device), or by a machine (e.g., a server, LPTM, etc.).

[0037] The term ‘streaming content’, or ‘streaming’, may refer to data provided as a stream of data elements being sent and / or received at a predetermined repetition. For example, video may be sent and / or received at the frequency of 30 frames per second (fps), where each frame may include the same number of pixels, and each pixel may include the same number of bytes. The number of fps here (temporal resolution) is arbitrary as well as the number of pixels in a frame (spatial resolution) and number of bits in a pixel (color resolution).

[0038] The term ‘application’ may refer to a software program running on, or executed by, one or more processors of a computing devices, and particularly by a mobile computing device such as a mobile telephone, a tablet, a smartphone, etc., as well as any other mobile or portable computing facili ty. The term ‘mobile application’ may refer to an application executed by a mobile computing device.

[0039] The term ‘large language model’ (TLM) or large pre-trained model (LPTM) may refer to any type of pre-trained model that may analyze content and / or generate content.

[0040] The term ‘interaction’ between pre-trained model and human may refer to a back-and-forth exchange of generated data between a human and a machine (e.g., LPTM). The term ‘iteration’ (when referring to interaction) may refer to a single interaction while the tenn “session’ may refer to a prolonged interaction. The terms ‘machine’, ‘model”, ‘pre-trained model’, and ‘LLM’ may be used interchangeably. It is appreciated that the system herein may be able to leverage previous interactions with a user (or users) to conduct a better current interaction with the user / s.

[0041] The tenn ‘system prompt’ may refer to any prompt that is fed to a large pretrained model (LPTM) prior to (or with) a user prompt. The term ‘system prompt’ may also be known as a “model prompt”, and a “technical prompt”. All system prompts may be provided to the LPTM in every interaction with the LPTM. Large pre-trained models (LPTM) currently necessitate that the data produced during the interaction with the LPTM is returned to the LPTM on each iteration of the interaction. Therefore, the data being input to the LPTM in each iteration grows considerably over time. Hence, all the data is analyzed again during each iteration. Obviously, this increases the processing requirements and thus increases the response time to the user and the cost of interaction. Every iteration is slower and costly than the previous iteration.

[0042] This problem increases with media that is 'heavy' (e g., by number of bytes) by its nature, such as visual data, audio, and video. The purpose of the memory optimization system is to optimize the use of the LPTM by eliminating at least some of the repetition w ith an emphasis on visual data and audio data.

[0043] Reference is now made to Fig. 1, which is a simplified block diagram of an LPTM optimization system 10, according to one exemplary embodiment.

[0044] As seen in Fig. 1. the LPTM optimization system 10 may include a large language model (LLM) 11 , communicatively coupled to a memory optimization system 12, which is communicatively coupled to a user input device 13.

[0045] It is noted that the terms ‘large language model 11 ', LLM 11, and LPTM 11, are interchangeable and may refer to any pre-trained artificial intelligence (Al) system. It is appreciated that input device 13 may be a client device such as a terminal, PC, smartphone, etc. or a (network) server, or both. In this respect, a user may input some of the data such as user prompts from a client device, and then feed audio and / or visual (including video) data from a netwok server. Moreover, the client device may include an artificial intelligence (Al) system that may interact with LPTM 11 via memory optimization system 12.

[0046] In this regard, Fig. 1 may represent two Al systems (denoted 11 and 13) that may interact with the mediation of memory optimization system 12. Hence the process of memory' optimization system 12 (as will be further disclosed below) may be used by both Al systems. Alternatively, each Al system (11 and 13) may use its own memory optimization system 12. For that matter, each Al system (11 and 13) may be denoted as an input system providing input data. The user inpur device 13 may be a client device and may include a keboard 14, a microphone 15, and a camera 16 to produce text input, audio input and visual (e.g, a picture, graphics, etc.) or video (e.g. streaming visual, animated graphics, streaming video, etc.) input, respectively. The input device 13 may also include a pointing device (not shown) and a variety of output devices 17, such as a speaker and a display device.

[0047] The memory' optimization system 12 may also include a local storage 18 and a database management system 19. The term ‘database’ hereinafter may refer to any type of database such as a hierarchical database, a network database, a relational database (such as SQL), a NOSQL database, a graph database, a vector database, an ephemeral database (e.g., Memcashed , Redis, etc.), a converged database, etc.

[0048] It is appreciated that DBMS 19 may include any number of DBMSs of various types as listed above. Each such database of DBMS 19 may include a search engine to search for and relieve particular records of data stored in storage 18. For simplicity, the term ‘record’ (or ‘object’) may refer to any type of data or media. It is appreciated that each such record of each of the databases may include a timestamp of its creation (e.g., by a respective input device) and / or reception by memory optimization system 12.

[0049] Reference is now made to Fig. 2, which is a flow chart of a main process 20 of memory' optimization system 12. according to one exemplary embodiment.

[0050] As an option, the flow chart of Fig. 2 may be viewed in the context of the previous Figures. Of course, however, Fig. 2 may be viewed in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below.

[0051] As shown in Fig. 2, main process 20 may start with action 21 by receiving the user’s input and storing it in DBMS 19. As shown in Fig. 1, the user’s input may be of various media (not detailed in Fig. 2), and therefore the action of storing may involve various databases. For example of saving data in more than one type of database may be saving textual data or links to larger media types stored in a file based system storage such as images. For example, storing audio, visual, and / or video data may involve creating and / or storing embedding data in a vector database. For example, saving data in an ephemeral database. The terms "embedding’, ""embeddings’, and "embedding data’ may refer to perceived visual embedding (e.g. image classifier embedding, etc.

[0052] Main process 20 may then proceed to action 22 to wait for a trigger. A trigger may be an event, which may also be a time-out, that may initiate interaction with LPTM 1 1 . For example, the trigger may be a user text input, for example followed by ‘enter’ (e.g., a carriage return character), and / or a time-out. For example, the trigger may be a streaming data followed by an end-of-information signal and / or a time-out. Any type of medium may have one or more delimeters and / or a time-out value.

[0053] Checking for a trigger may be performed by examining incoming media from the user and utilizing the same or another model to understand if the user is expecting a response from the machine. A response from the machine may be required because a strong memory was found, which may be related to the media currently provided.

[0054] Another example may be receiving textual information pertaining to a question. Another example may be visual and / or textual data that has a high probability of relating to an earlier interaction. Yet another example may be visual and / or auditory information provided where analysis suggests a heightened emotional state and a requirement in instructional system data to respond when detected, etc.

[0055] If a trigger is identified, main process 20 may proceed to action 23 to prepare a data batch. The term ‘data batch’ (or ‘batch data’ or simply ‘batch’) may refer to data provided to LPTM 11 at each iteration of the interaction between memory' optimization system 12 and LPTM 11. Such data batch may include user prompt(s), system prompt(s), textual data, audio data, visual data (including video), etc.

[0056] Data batch may be prepared by action 23 from data stored in database 19 and / or storage 18. Action 23 may retrieve from database 19 and / or storage 18 a data window of most recent data. The tenn ‘data window’ may refer to the size of the data to be retrieved, where the size may be given in bytes, or seconds (of audio and / or video), or pictures, or frames (of video), etc.

[0057] For example, the data batch may include the data window plus the user input received in the most recent action 21. Main process 20 may then proceed to action 24 to communicate the data batch to LPTM 11. Data batch preparation 23 is further described below with reference to Fig. 3.

[0058] In this respect the actions 21. 23, and 24 may include the steps of:

[0059] Obtaining user data from a user.

[0060] Storing the user data in at least one database.

[0061] Creating a data window, the data window comprising a predefined amount of data recently acquired by the at least one database.

[0062] - Providing the data window to a large pre-trained model (LPTM),

[0063] It is appreciated that after actions 21 to 23 are repeated for several times (namely main process 20 is repeated for several times) the above sequence of steps is repeated several times and thus the data batch may include a sequence of data windows, each data window associated with one sequence (repetition) of actions 21 to 23. Each such data window may include user provided data, which may include audible and / or visual data. Each such data window may include: a predefined number of seconds preceding a time-stamp associated with the data recently acquired; and / or a measure in seconds of the amount of data recently acquired following the time-stamp.

[0064] Main process 20 may then proceed to action 25 to interact with LLM 11 as further described below with reference to Fig. 4. When LPTM 11 provides a response to the user, main process 20 may proceed to action 26 to receive the response, store the LPTM response in database 19 and / or storage 18 (action 27) and communicate the LPTM response to the user (action 28).

[0065] Reference is now made to Fig. 3, w hich is a How chart of data batch preparation process 23 of memory' optimization system 12, according to one exemplary embodiment.

[0066] As an option, the flow chart of Fig. 3 may be viewed in the context of the previous Figures. Of course, however, Fig. 3 may be viewed in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below. Data batch preparation process 23 of Fig. 3 may be viewed as action 23 of Fig. 2. Data batch preparation process 23 may start with action 29 by selecting a data type or a medium, which may be used interchangeably to denote a medium or a data type such as text, numeric, alphanumeric, audio, video, still pictures, etc. It is appreciated that each of these data types may have its own database (or more than one database), which may have a search engine.

[0067] It is appreciated that each such medium or data type may also have a particular data window (as explained above). The size of the data window to be retrieved from the respective database of the respective data type or medium may be given in number of bytes, characters, digits, pictures, frames, seconds, etc.

[0068] Data batch preparation process 23 may then proceed to action 30 to determine if such data winbdow is defined for the particular data type / medium / database. If the data type is not defined (action 31), the entire data for the particular type / medium / database is collected to be added to the data batch. If the data type is defined (action 32), the respective amount of most recent data for the particular type / medium / database is collected to be added to the data batch.

[0069] Data batch preparation process 23 may then proceed to action 33 to add the data collected in actions 31 or 32 to the memory of the data batch. It is appreciated that the data collected in actions 31 or 32 is made of a sequence of records (of the respective data type or medium), where each record may include a timestamp of the time of creation of the record by a respective input device, and / or the time of reception of the record by the respective database. In action 33 each such collected record is added to the data batch according to the record's timestamp.

[0070] In this respect, if the LPTM responds with a request for required data, the data batch preparation process 23 may execute the following steps:

[0071] Compose a search query in the database for the required data.

[0072] - Add the data acquired from the database responsive to the query to the data window, to form an extended data window.

[0073] Provide the extended data window to the LPTM. The sequence of actions 29 to 33 may be repeated until all data types and media are processed (action 34). At this point the memory of the data batch may include a long sequence of records of various data types and media ordered according to their respective time stamps.

[0074] Data batch preparation process 23 may then proceed to action 35 to add to the memory7of the data batch the current (most recent) data received, if such data exists.

[0075] Reference is now made to Fig. 4. which is a flow chart of interaction module 25 of memory optimization system 12, according to one exemplary’ embodiment.

[0076] As an option, the flow chart of Fig. 4 may be viewed in the context of the previous Figures. Of course, however, Fig. 4 may be viewed in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below.

[0077] Interaction process 25 of Fig. 4 may be viewed as action 25 of Fig. 2. Interaction process 25 may start with action 36 by receiving the LPTM (item 11 of Fig. 1) response to the data batch communicated in action 24 of Fig. 2. Interaction process 25 may then proceed to action 37 to analyze the LPTM response into three options.

[0078] The first option (action 38) is where the response includes content newly generated by the LPTM. The second option (action 39) is where the response only includes an acknowledgement of the content received by the LPTM. In both the first and the second options the Interaction process 25 ends.

[0079] The third option (action 40) is when the LPTM response includes a requirement for more data. Interaction process 25 may then proceed to action 41 to determine what type of data (or medium) is required, which database to querry, to compose a search query into the relevant database and submit it. It is appreciated that the query may include embedding data.

[0080] It is appreciated that the search into the relevant database is made from the newest record and backward in time (according the the timestamp of each record). Therefore, the content derived from the database is expected to include the newest content (most recent) that the query may retrieve. Interaction process 25 may then proceed to action 42 to determine if the response from the database is successful, in the sense that the data retrieved from the database is in line with the LPTM requirements. If the response is deemed successful interaction process 25 may proceed to action 43 to compose the data collected from the database as an input to the LPTM, and communicate it to the LPTM.

[0081] If, in action 42, the response from the database is deemed unsuccessful, interaction process 25 may try again (action 41) for a limited number of retries (action 44).

[0082] It is appreciated that when interacting with the pre-trained model (such as LLM) interaction process 25 may provide the pre-trained one or more instructions, for example in the form of system prompts, informing the pre-trained model that the user prompt we are providing might not have all the data from previous interactions in this session(s).

[0083] Hence, when the system prompt(s) and the user prompt(s) are provided to the pre-trained model the pre-trained model is instructed to respond with one of three types of generated data as descrubed above with reference to action 37 of Fig. 4

[0084] If the model has enough information to generate data in response to the data provided by the system “New generated data7’ in the form of text, image, video, etc (action 38) is provided to the system and the system will communicate the data to the user.

[0085] If the data received is sufficient but the pre-trained model recognizes that it may use more data from the user, the pre-trained model may be instructed (e g., by way of a system prompt) to respond with an acknowledgement and wait for the next part of a conversation (interaction).

[0086] If the pre-trained model recognizes that the current batch of data provided refers to information generated in the past that was not included in this batch of data provided the pre-trained model is configured (as required in a system prompt) to respond with a “Search required” response and provide what to search for. It is appreciated that the batch of data may include not only user generated data, but also model generated data, and the pre-trained model may note that data it generated is missing.

[0087] If the search query finds relevant data (and / or media) the data (media) may be added to the current batch data and may be provided to the pre-trained model, so that the next LPTM response may generate new content. In some cases interaction process 25 may loop for for a limited number of times (actions 41, 42, and 44). At this pomt we may communicate to the user that more data is required.

[0088] LPTM optimization system 10 is described above with memory optimization system 12 as an intermediating system between the input system (client device 13) and the pre-trained system (LPTM 11). However, memory optimization system 12 can also be described as part of the pre-trained system (LPTM 11) enabling LPTM 11 to optimize the use of its memory by creating and managing a long term memory (storage 18, DBMS 19) and a short-term memory (data window, batch data).

[0089] In this respect the process may have the following steps:

[0090] - obtaining input data from an input system; storing the input data in at least one database; creating a data window, the data window comprising a predefined amount of data recently acquired by the at least one database;

[0091] - receiving the data window by the LPTM,

[0092] - determining, by the LPTM, that further data is required; composing a search query in the at least one database for the required data; adding the data acquired from the database responsive to the query' to the data windoyv to form an extended data window; and

[0093] - receiving the extended data window by the LPTM.

[0094] It is appreciated that certain features, which are, for clarity7, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features, which are, for brevity, described in the context of a single embodiment, may7also be provided separately or in any7suitable subcombination. Although descriptions have been provided above in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims. All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation, or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art.

Claims

CLAIMS:What is claimed is:

1. A computer-implemented method for optimizing interaction with a large Pre- Trained Model (LPTM), the method comprising: obtaining input data from an input system; storing the input data in at least one database; creating a data window, the data window comprising a predefined amount of data recently acquired by the at least one database; and providing the data window to the large pre-trained model (LPTM).

2. The method according to claim 1, additionally comprising: if the LPTM responds with a request for required data: composing a search query in the at least one database for the required data; adding the data acquired from the database responsive to the query' to the data window to form an extended data window; and providing the extended data window to the LPTM.

3. The method according to claim 1, additionally comprising: if the LPTM responds with LPTM provided content: adding the LPTM provided content to the at least one database; and communicating the LPTM provided content to the user.

4. The method according to claim 1, additionally comprising: the user data comprises at least one of audible and visual data; the predefined amount of data recently acquired comprises at least one of: a predefined number of seconds preceding a time-stamp associated with the data recently acquired; and a measure in seconds of the amount of data recently acquired following the time-stamp.

5. The method according to claim 1, additionally comprising: the user data comprises at least one of audible and visual data; the storing the user data in the at least one database includes storing at least one embedding data associated with the at least one of audible and visual data; and a search query comprises the at least one embedding data.

6. A computer program product embodied on a non-transitory computer readable medium, including instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising: obtaining input data from an input system; storing the input data in at least one database; creating a data window, the data window comprising a predefined amount of data recently acquired by the at least one database; and providing the data window to a large pre-trained model (LPTM),7. The computer program product according to claim 6, additionally comprising: if the LPTM responds with a request for required data: composing a search query in the at least one database for the required data; adding the data acquired from the database responsive to the query to the data window to form an extended data window; and providing the extended data window to the LPTM.

8. The computer program product according to claim 6, additionally comprising: if the LPTM responds with LPTM provided content: adding the LPTM provided content to the at least one database; and communicating the LPTM provided content to the user.

9. The computer program product according to claim 6, additionally comprising: the user data comprises at least one of audible and visual data; the predefined amount of data recently acquired comprises at least one of: a predefined number of seconds preceding a time-stamp associated with the data recently acquired; anda measure in seconds of the amount of data recently acquired following the time-stamp.

10. The computer program product according to claim 6, additionally comprising: the user data comprises at least one of audible and visual data: the storing the user data in the at least one database includes storing at least one embedding data associated with the at least one of audible and visual data; and a search query comprises the at least one embedding data.

Images