Visual memory in autonomous agents

Abstract

Embodiments described herein provide a method by which an autonomous agent (which may be a situationalized and materialized agent) learns new visual objects, recognizes those objects within the same “runtime” / session / interaction, and enables the training of a long-term visual memory model for those objects. The visual memory system stores templates of objects and compares images to these templates. The choice of which objects to compare may be determined by internal or external signals (of the agent). The visual memory system stores new and unrecognized objects as “templates” and returns whether any region of an image is likely to contain a match with any of the stored templates. In this way, one-shot visual learning is achieved.

Description

Detailed Description of the Invention

[0001] 〔Technical Field〕 Embodiments of the present invention relate to computer vision and situated and embodied computational agents. More specifically, but not limited to, embodiments of the present invention relate to visual memory in autonomous agents.

[0002] 〔Background Art〕 In computer vision tasks, deep neural networks (e.g., Convolutional Neural Networks (CNNs) and Vision Transformers) are used. CNNs are a type of widely used deep learning model and have been shown to be very effective in object recognition and classification tasks. These models are designed to mimic the architecture of the "what" pathway and consist of stacking layers of neurons to process increasingly complex visual features.

[0003] Object detection models are trained using a certain class of objects, but they may not be able to recognize and accurately classify all objects presented to the object detection model. It may be desirable in an embodied interactive virtual agent to knowingly interact with unrecognized new objects, learn about those new objects, and / or consolidate the learned content about new objects in an efficient and appropriate manner. Some few-shot learning / one-shot learning methods have been proposed, but these are trained or manually designed to preprocess input data into a form more suitable for matching and searching.

[0004] Catastrophic forgetting refers to the phenomenon where a neural network trained to perform multiple tasks loses its ability to accurately perform previous tasks when trained to perform a new task. This is one of the major challenges in artificial intelligence because it limits the ability of machine learning models to continuously learn over time. When a model learns a new task, the weights associated with the previous task are adjusted, which can cause the model to forget what it has learned so far. Attempts have been made to address this, including rehearsal, regularization, parameter freezing, and the use of multiple models for various tasks.

[0005] A sequential learning problem refers to the challenges that arise when training a machine learning model to learn from data presented in a sequential order. In sequential learning, the model needs to update its knowledge and make predictions as new data becomes available. The challenge is that the model must balance the knowledge it has learned from previous data with the newly received information. This is because the data distribution in a sequential learning problem can change over time, which can affect the model's performance.

[0006] A new visual object detection system is needed that addresses at least some of the limitations of conventional technologies.

[0007] [Summary of the Invention] The object of the present invention is to improve visual memory in autonomous agents, or at least to provide a useful alternative to the public or industry. The improvements described herein support real-time one-shot visual learning, enable real-time sequential learning and catastrophic forgetting, and result in improved performance, efficiency, and generalization ability.

[0008] [Brief explanation of the drawing] Figure 1 shows the visual processing system.

[0009] Figure 2 shows an interface illustrating the results of the image matching algorithm.

[0010] Figure 3 shows three objects in the visual scene and their corresponding heatmaps.

[0011] Figure 4 shows the first output of the visual memory system.

[0012] Figure 5 shows the second output of the visual memory system.

[0013] [Description of Embodiments] (overview) Embodiments described herein provide a method by which an autonomous agent (which may be a situationalized and materialized agent) learns new visual objects, recognizes those objects within the same “runtime” / session / interaction, and trains a long-term visual memory model about those objects. The visual memory system stores templates of objects and compares images to these templates. The choice of which objects to compare may be determined by internal or external signals (of the agent). The visual memory system stores new and unrecognized objects as “templates” and returns whether any region of an image is likely to contain a match with any of the stored templates. In this way, one-shot visual learning is achieved.

[0014] Therefore, an artificial intelligence agent can be presented with an object only once, store a template of that object, and then query that stored template to recognize the object in the future (including within the same execution instance of the AI ​​agent). This enables on-the-fly, one-shot visual learning because any template image of any part of the scene can only be cached once. By using an image matching system, future areas of the scene that match the template can be identified, allowing the template image (or part of the scene) to be identified in future images.

[0015] Scenes are queried using an image matching system. This enables reliable detection of the same object across different parts of the scene. Labeled data can then be generated using this system, and bounding regions (e.g., bounding boxes) can be added to the scene and sent to a long-term memory storage system (e.g., an object detector) for use in training a visual model.

[0016] In the post-processing implementation of image segmentation, classification information is grouped into a vector of multiple classification types. This classification type includes the class name, bounding box, name ID, confidence level, objectness, forbear score (degree of overlap with forbears), and whether the classification information was generated by an object detector or a template matcher. Only classification information that reaches a specific confidence / objectness threshold is included in this vector.

[0017] Figure 1 shows a visual memory system according to one embodiment. An image matching system 108 and an object detector 110 are used for segmentation 112 of an input scene 102. A forebear 104 may represent the area of ​​focus of an autonomous agent employing the visual memory system. A learning signal 106 may be provided so that a new template is stored in the image matching system 114 when the learning signal is high (in a state).

[0018] Figure 2 shows the interface illustrating the results of the image matching algorithm. The top left box (output_image) represents the snake template. The image matching algorithm compares the block image (image_data.data) in the top right with the template. The bottom right image (response.data) shows the output from the template matching algorithm, with brighter colors indicating a higher degree of match according to the template matching algorithm. Note that the areas in the block image where the snake block matches the snake template are displayed much brighter. This indicates the presence of the template object within that area.

[0019] Figure 3 is another interface showing the results of the image matching algorithm. On the left, three templates are shown: a snake 302, a horse 304, and the letter "A" 306. The image matching algorithm compares each template to the input scene 300. The three outputs 312, 314, and 316 shown on the right correspond to each object template, respectively. Output 314, regarding the comparison with the horse template 304, has significantly less white space. This is because there is no good match for this template in the input scene 300. In 312, the areas in the input image 300 where the snake appears are highlighted in white, and similarly in 316, the areas of the input scene where the "A" block 306 appears are highlighted in white.

[0020] Figure 4 shows the first output of the visual memory system. A zebra, a second zebra, and a giraffe have been detected. Another object (i.e., the play button) is also present in the autonomous agent's visual stream, but has not yet been detected. Figure 5 shows the second output of the visual memory system, where the play button has been identified and added to the image matching system (in this case, as a QATM object). The play button in the context of the rest of the scene can now be added as training data to a trainable object detector.

[0021] Thus, an artificial intelligence agent can be driven using two parallel visual systems: (1) an image matching system that can learn and identify new objects on the fly (during execution) by finding which parts of the input image / scene match a pre-trained template, and (2) an object detector that may be pre-trained to identify a wide range of objects and can continuously output all objects it recognizes.

[0022] Subsequently, the object detector can be trained to incorporate the new objects added to the image matching system using training examples of those new objects acquired within the visual stream of the artificial intelligence agent.

[0023] (Image matching system) The image matching system performs one-shot visual learning. When a learning signal is received, it triggers the transmission of an input to the image matching system, and that image is saved as a template.

[0024] Input to the image matching system The image matching system may accept one or more of the following as input: • An image of the entire scene. This may include everything contained within one or more "visual fields" of the materialized agent. In one embodiment, this is a camera stream. In other embodiments, this may be an input from a digital user interface, a computer application, or any other suitable input. • Bounding regions (e.g., bounding boxes) (which may be generated by the object detector or "Fovea"). One or more bounding regions (e.g., bounding boxes) may be created around a detected object. In one embodiment, bounding regions (e.g., bounding boxes) are selected from all bounding regions (e.g., bounding boxes) detected by the object detector on the scene according to certain criteria. For example, when sending an image to a visual memory system, the bounding region (e.g., bounding box) with the greatest overlap with the Fovea is selected. A certain threshold may be imposed on the degree of overlap between the Fovea and the bounding region (e.g., bounding box) before sending the selected bounding region (e.g., bounding box) to the visual system for further processing. • Fovear position • Object detection labels

[0025] Image matching heatmap check: Returns a heatmap for each template. When a queried input (e.g., an image of an entire scene) is provided, the image matching algorithm searches for the saved templates and determines whether there is a match with the already saved templates. When the image matching system is queried about a scene, a heatmap is returned for each saved template for each of the saved templates (images already saved in the image matching system). In other words, the frame to be inferred is compared with each template, and a heatmap is returned for each template. The heatmap may be passed through a box filter twice to identify the "peaks" of the heatmap. The maximum intensity value of the heatmap after filtering is used as the confidence level, and the position of that intensity value is estimated to be the center of the object. To generate a bounding region (e.g., a bounding box) for image segmentation, the dimensions of the template image are used, and a bounding region (e.g., a bounding box) is placed around the estimated center. The template ID is used as the class label.

[0026] The entire scene image may be queried by a template matcher (e.g., QATM) for a match with any template on the server. When a heatmap is returned, the system compares the average saliency within the bounding region (e.g., a bounding box) with the average saliency of the entire scene. If this ratio reaches a pre-determined template match threshold, the object is classified as matching the template.

[0027] Generation of Output: In one embodiment, the image matching system generates a class label, which is a unique identifier for the class of the object, from a set of unique identifiers of each object known by an object detector, in addition to the objects saved in the image matching system.

[0028] In another embodiment, alongside the learning signal, the user (or someone else) provides a class label, which is a unique identifier for the object's class. This may be natural language representing the object class (e.g., cup, chair, person).

[0029] The image matching system may generate output (training examples) for future training or for immediate use within the runtime of the materialized agent. The following output may be generated, saved, and used to train an object detector. This object detector may be trained during the integration phase: • Scene images • Input bounding area (e.g., bounding box) • A text label associated with a bounding region (e.g., bounding box). For example, object_X, where X = class label from the image matching system.

[0030] (Fobea) An autonomous agent may include a fovea simulation. A fovea is the region that an materialized agent is paying attention to / gazing at at any given time. A fovea may be defined by a bounding region (e.g., a bounding box). The visual processing system may be configured to return the fovea bounding region (e.g., a bounding box) so that it can maintain information about the location the autonomous agent was gazing at, associated with image segmentation.

[0031] The degree of overlap between each bounding region (e.g., bounding box) obtained from image segmentation and the fovea bounding region (e.g., bounding box) is calculated. If the agent is fixated on an object, the overlap calculation will return 1 if the object is entirely within the fovea (even if it has a very small bounding box). The result of this overlap calculation is the "fovea score". The fovea score is associated with each classification. This is used to determine whether an object is being noticed by a fovea at any given time, and which objects are being noticed by a fovea at any given time.

[0032] An autonomous agent or avatar can be associated with a field ("Fovea class") that represents the object the agent is fixated on. The Fovea class is determined by the highest Fovea score (degree of overlap with the Fovea) among the classification vectors (uncertain classifications have already been filtered out). If the highest Fovea score exceeds a certain threshold, that class becomes a Fovea class; otherwise, there is no Fovea class (in other words, the system determines that the agent is not fixated on the object; for example, the agent may be looking at the background or empty space).

[0033] (Output to the situation) The agent may have a representation of "Situations," that is, a list of things that exist in the scene and their locations. Image segmentation may provide output to the situations in the form of the presence or absence of mapped objects in the scene, as well as the location and center of each of those mapped objects.

[0034] (Output to the sampling map) A salience map is an agent's representation of the screen in terms of importance. The salience map can determine where the agent's attention and focus lie. Examples of features that can be treated as either salient or non-salient include the recognition results of objects or images, or the recognition results of subclasses of objects or images. Output from a visual processing system can be provided to the salience map. This salience map can then bias or direct the autonomous agent's attention / foreground based on any number of factors. Thus, "objects" are provided as a set of inputs that can increase the salience of areas within a scene. An "object saliency map" can highlight the scaled central locations of objects identified during segmentation as salient areas.

[0035] (Consolidation process) Image matching systems are used to generate training data for more generalized visual models and to integrate objects learned during the operation of interactive agents into long-term visual memory as objects (modeling consolidation).

[0036] In training object detection, the custom training process may be fine-tuned on a new dataset using rehearsals. At each epoch, the training process trains on a random subset of data from a pre-trained set and combines it with the entire new dataset. This is to avoid catastrophic forgetting of previously learned data and to circumvent the sequential learning problem. Any appropriate integration process may be used.

[0037] Integration can be triggered in any appropriate way. For example, integration can be triggered as follows: • Manually · Every time a new object is visible • After the "session" in which the user interacts with the agent After a predetermined number of new objects have been learned • After a certain amount of time has passed

[0038] Training set for integration Even if an object is a second encounter with a new object in short-term memory, it still becomes a training item for the object detector (having the same label). Furthermore, even objects that are already classifiable by the object detector (i.e., objects already trained via the object detector) become training data if they appear in images that are already training data. As a result, the object detector continues to recognize these objects even after integration.

[0039] Integration in progress Memory consolidation may be performed while the agent is “sleeping,” in a non-running (operating) state, or in a state where it is not possible to query its visual memory system. In some embodiments, duplicates of the object detector may be created so that one version is trained while the other is queried, and then the model is replaced with the latest version.

[0040] (Integration with runtime) Whether objects are trained by an image matching system or an object detector, their identification downstream and integration with the features (functions) of an autonomous agent (e.g., memory mapping, context, splendor) may be handled in the same manner.

[0041] Object Detector An object detector is a model designed to detect objects. These models are known for their fast inference speeds, which makes them suitable for speed-critical real-time applications. Examples include YOLO, SSD (Single Shot Detector), EfficientDet, and YOLOX. Single-shot detectors are a popular choice for real-time object detection applications where fast inference is crucial (e.g., autonomous vehicles, video surveillance, and robotics). Single-shot detectors divide an image into a grid of cells and predict the position, class, and size of objects within each cell. This is typically achieved using anchor boxes. Anchor boxes are predetermined bounding regions (e.g., bounding boxes) of various sizes and aspect ratios that the model uses to predict the position and size of objects.

[0042] Given a scene (input frame), the object detector may return the following: • The bounding area (e.g., bounding box) around the identified object. • Objectness score (how confident the object detector is that the bounding region (e.g., bounding box) actually contains an object). • Class label (what exists within the bounding region (e.g., bounding box)). • Class confidence (how confident YOLOX is about that class).

[0043] The object detector may have a limit on the maximum number of bounding regions (e.g., bounding boxes) it returns per frame (e.g., 10).

[0044] YOLOX In one embodiment, the single-shot detector used in a visual memory system is YOLOX (You Only Look Once X). The YOLOX model is based on a single-shot detector architecture, meaning it processes the entire image only once to detect objects in real time. YOLOX achieves this by using a set of multiple anchor boxes to predict the object's position, scale, and aspect ratio. Furthermore, YOLOX incorporates several innovations to improve the accuracy and efficiency of object detection:

[0045] Decoupled Head: The head of a YOLOX model is decoupled from the backbone. This increases flexibility in model design and allows for the use of larger backbones for improved accuracy.

[0046] Cross-Stage Partial Network (CSPNet): This is a novel network architecture that improves information flow and feature reuse by using parallel partial networks and cross-stage fusion operations.

[0047] Scaled YOLOX: This is a set of multiple models with various model sizes that can be scaled up or down according to different performance requirements. This allows for more efficient use of computing resources, enabling YOLOX to perform well on a variety of devices.

[0048] Adaptive Training Sample Selection (ATSS): This is a sampling method that selects training samples based on difficulty and importance, which improves the model's ability to handle small objects and dense scenes.

[0049] (Shared core) In one embodiment, elements of an image matching model and elements of an object detection model may be linked. For example, a shared CNN core of the image matching model may be used for feature processing in the object detector.

[0050] (Image matching system / algorithm) Image matching algorithms are a type of computer vision algorithm used to detect similarities or correspondences between two or more images. The goal of image matching is to determine whether two images contain the same object, scene, or pattern, and to estimate the transformations that relate them. Several types of image matching algorithms exist, including feature-based matching, template matching, and deep learning-based matching.

[0051] Feature-based matching matches keypoints or features between images based on local appearances such as edges, corners, or blobs. Examples of feature-based matching algorithms include SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF).

[0052] Template matching: This method matches a predetermined template or pattern with an area of ​​an image to find where the template appears. Examples of image matching algorithms include NCC (Normalized Cross-Correlation) and QATM (Quality-Aware Image Matching).

[0053] Deep learning-based matching uses convolutional neural networks (CNNs) to extract features from images and learns how to match them using supervised or unsupervised methods. Examples of deep learning-based matching algorithms include the Siamese network, which learns a similarity metric between multiple image pairs, and SuperPoint, which uses a CNN to simultaneously detect keypoints and descriptors.

[0054] Q ATM QATM (Quality-Aware Image Matching) is designed to match images with different resolutions and aspect ratios by estimating the quality of image regions and performing image matching on the regions with the highest quality. QATM works as follows: First, it divides the input image into non-overlapping blocks of different sizes. Next, it uses a quality estimator to assign a quality score to each block based on the local contrast and structure of the image. Blocks with high quality scores are considered to contain important image features and are used for image matching.

[0055] For each high-quality block, QATM performs image matching by sliding the template over the block and calculating a similarity score at each position. The position with the highest similarity score is considered the matching position with the template.

[0056] Furthermore, QATM includes several optimizations to improve the accuracy and efficiency of image matching. For example, QATM employs a multiscale approach to match templates of different sizes and also includes a non-maximum suppression step to remove duplicate matches.

[0057] In the context of the present invention, image matching algorithms such as QATM may be used to perform one-shot learning of visual objects that are not recognized by the object detector. For the purposes of the visual memory systems described herein, image matching algorithms store templates, which are later used to query a scene to determine whether any object in the visual scene matches one of the stored templates. Given an input template, the image matching system may return a "heat map" of the scene or input image. The heat map returns a scene or image with areas highlighted to varying degrees / intensities, reflecting how well those areas match the input template. In other words, it is possible to query the location of template match candidates in the image and the matching intensity of those candidates.

[0058] (3D object template) If necessary, the present invention may be adapted so that the image matching algorithm can store and recognize 3D objects in various orientations and scales. For example, multiple templates may be stored for each template, and / or a spatiotemporal consistency model may be employed to improve image matching. For example, if object 1 exists at a similar location and time to object 2, they are likely to be the same object (especially if the object is being tracked)—the threshold for what is considered a "match" may be reduced in such a situation, for example.

[0059] (Simultaneous training of object detectors by multiple agents, instances, or users) In one embodiment, the memory store may collect data from any number of instances of running autonomous agents, which may be used to train one or more detection algorithms or trainable object detectors described herein.

[0060] The memory store may be queried to retrieve a subset of the data so that object detection (or other visual processing algorithms) can be trained in a specific way (e.g., by user, time of day, context, or by any other appropriate parameter). For example, when purchasing a car, the algorithm might be trained on all the objects that people interact with.

[0061] Patent application WO2021005540A2, owned by the present applicant and incorporated herein by reference, discloses how the output of a visual processing system may be stored in a memory database. This memory database is accumulated from the real-time experiences of an autonomous agent (which may be interacting with a human user or a computer system) during its operational use. The memory database stores memories experienced by the autonomous agent. These memories may include metadata, or additional data relating to the experience stored in the memory database, such as the time of event (timestamp), the GPS location of the event, or any other contextual information related to the experience. Further metadata stored by the memory store may include data relating to the user, the running "agent" or its version, the running instance, the context, or any other appropriate metadata.

[0062] (System Overview) The visual short-term memory system stores object templates and compares images to these templates. The system stores new and unrecognized objects as templates and determines whether the object being looked at matches a stored template. This enables "one-shot visual learning": the materialized agent or avatar only needs to be presented with the object once (for the template to be stored), and thereafter, the materialized agent or avatar can query the stored template to recognize the object in the future.

[0063] Therefore, the images used by the image matching system can be updated and used in real time by the image matching system while the user is interacting with the system, and the learned content can then be integrated into the object detector by training the object detector using the objects identified by both the image matching system and the pre-trained object detector. The embodiments described herein enable real-time sequential learning based on the experience of an agent-based AI system.

[0064] 〔interpretation〕 The methods and systems described herein are available in any suitable electronic computing system. According to the embodiments described below, the electronic computing system utilizes the methods of the present invention using various modules and engines. The electronic computing system may include at least one processor, one or more memory devices or interfaces for connecting to one or more memory devices, input and output interfaces for connecting to external devices to enable the system to receive and process instructions from one or more users or external systems, a data bus for internal and external communication between various components, and a suitable power supply. Furthermore, the electronic computing system may include one or more communication devices (wired or wireless) for communicating with external and internal devices, and one or more input / output devices such as a display unit, pointing device, keyboard, or printing device. The processor is configured to execute steps of a program stored as program instructions in a memory device. The program instructions enable the methods of carrying out the invention described herein. The program instructions can be developed or implemented using any suitable software programming language and toolkit, such as a C-based language and compiler. Furthermore, the program instructions can be stored in any suitable way that is transferable to the memory device or readable by the processor. For example, this includes storage in a computer-readable medium. Computer-readable media are any suitable medium for physically storing program instructions, including, for example, solid memory, magnetic tape, compact disks (CD-ROM or CD-R / W), memory cards, flash memory, optical disks, magnetic disks, or any other suitable computer-readable medium. Electronic computing systems are configured to communicate with data storage systems or devices (e.g., external data storage systems or devices) to retrieve relevant data. It is understood that the systems described in this document include one or more elements configured to perform the various functions and methods described in this document.The embodiments described herein are intended to provide readers with examples of how various modules and / or engines constituting elements of a system are interconnected to implement functions. Furthermore, the embodiments herein describe how the steps of the methods described herein are performed, along with system-related details. Conceptual diagrams are provided to show readers how various data elements are processed at different stages by various modules and / or engines. The arrangement and configuration of modules or engines may be modified as appropriate according to system and user requirements, and various functions may be used by modules or engines different from those described herein, and specific modules or engines may be integrated into a single module or engine. The modules and / or engines described herein are understood to be implemented and provided with instructions using any appropriate technical form. For example, a module or engine may be implemented or created using any appropriate software code written in any appropriate language, which may be compiled to produce an executable program that can be run on any appropriate computing system. Instead of, or in combination with, this executable program, the module or engine may be implemented using any appropriate combination of hardware, firmware, and software. For example, parts of the module may be implemented using application-specific integrated circuits (ASICs), system-on-a-chip (SoCs), field-programmable gate arrays (FPGAs), or other suitable adaptable or programmable processing units. The methods described herein may be implemented using a general-purpose computing system specifically programmed to perform the steps described herein. Alternatively, the methods described herein may be carried out using specific electronic computer systems, such as data sorting and visualization computers, database query computers, graphical analysis computers, data analysis computers, manufacturing data analysis computers, business intelligence computers, and artificial intelligence computer systems.This computer is specifically adapted to perform the described steps on specific data obtained from an environment related to a particular field. [Brief explanation of the drawing]

[0065] [Figure 1] This shows a visual processing system. [Figure 2] This shows an interface that displays the results of the image matching algorithm. [Figure 3] This shows three objects in the visual scene and their corresponding heatmaps. [Figure 4] This shows the first output of the visual memory system. [Figure 5] This shows the second output of the visual memory system.

Claims

1. A method for training a trainable object detector, implemented by a computer, from a visual stream, i. The step of adding one or more objects in the visual stream to the image matching system; ii. A step of generating one or more training examples from one or more frames from the visual stream that include one or more of the objects added to the image matching system; and iii. A step of training the trainable object detector based on one or more of the training examples. Methods that include...

2. The method according to claim 1, wherein the image matching system is a template matching algorithm.

3. The method includes the step of identifying and classifying pre-trained objects appearing in the visual stream using a pre-trained object detector, The method according to claim 1, wherein one or more training examples also include pre-trained objects and classification information.

4. The method according to claim 3, wherein the object detector is a single-shot object detector.

5. The method according to any one of claims 1 to 4, wherein the step of adding one or more objects in the visual stream to the image matching system is associated with an action that causes an materialized agent to direct its attention to one or more of the objects.

6. A computer-based method for object detection, i. The process of receiving the input scene; ii. A step of identifying objects in the input scene using an object detection algorithm; iii. The step of identifying an object in the input scene using an image matching algorithm trained on an object identified in a previous frame of the input scene; and iv. A step of generating a representation of the image scene that includes objects identified using the object detection algorithm and the image matching algorithm. Methods that include...

7. A method for driving the visual attention of an materialized agent on an input scene, A step of detecting an object in the input scene using the method of claim 6, A step of using the representation of the generated image scene in the splendor map on the input scene to deflect the attention of the materialized agent toward the splendor region of the input scene, Methods that include...

8. A computer program, which, when the program is executed by a computer, includes an instruction causing the computer to perform the method described in any one of claims 1 to 7.

9. A computer-readable medium comprising, when executed by a computer, an instruction causing the computer to perform the method according to any one of claims 1 to 8.

Images