Systems, methods, and computer-readable media for object recognition

Abstract

A system, method, and computer-readable medium for hybrid cloud architecture for machine learning-based object recognition are disclosed. In one aspect, a system includes: one or more video-capable access points; and one or more processors configured to receive image data from the one or more video-capable access points; execute a first process at a first processor in the one or more processors to detect one or more objects of interest in the image data; generate vector IDs for the one or more objects detected in the image data; execute a second process at a second processor in the one or more processors to identify the one or more objects in the vector ID; and generate at least one offline track for the one or more objects based on statistical information associated with the identified one or more objects.

Description

[0001] Related application data

[0002] This application claims priority to U.S. Application No. 16 / 193,238, filed on November 16, 2018, which in turn claims priority to U.S. Provisional Patent Application No. 62 / 683,202, filed on June 11, 2018, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present technology relates to vector-based object recognition, and more particularly, to a hybrid cloud architecture for machine learning-based object recognition. Background Art

[0004] With the advancement of technology and the continuous growth of online commerce, many techniques have been developed to track users' online movements and activities (users' online trajectory) in order to understand their behavior, provide targeted advertising, etc. There are many limitations to applying the same concept to physical locations to track users' movements and activities within or through one or more physical locations (users' offline trajectory). For example, detecting consumers' movements throughout a store requires (1) installing a sufficient number of access points / beacons within the store and (2) consumers carrying mobile devices that are capable of communicating with the installed access points. In addition, available facial recognition-based methods are insufficient because they are computationally intensive due to the large amount of data to be processed, and they are unable to recognize different instances / captured images of a user in different poses, clothing, etc. and use them to create an offline trajectory of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In order to describe the manner in which the above and other advantages and features of the present disclosure can be obtained, a more particular description of these principles will be given by reference to specific embodiments of the principles briefly described above, which are illustrated in the accompanying drawings. Understanding that these drawings depict only exemplary embodiments of the present disclosure and are not therefore to be considered limiting of its scope, the principles of the present invention are described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0006] Figure 1 An example hybrid cloud architecture for image recognition according to one aspect of the present disclosure is shown;

[0007] Figure 2 According to one aspect of the present disclosure Figure 1 Components of the fog server at the fog layer;

[0008] Figure 3 According to one aspect of the present disclosure Figure 1Components of servers at the private platform layer of;

[0009] Figure 4 According to one aspect of the present disclosure Figure 1 Components of servers at the public platform layer of the;

[0010] Figure 5 An example method for developing a machine learning-based model for object recognition according to one aspect of the present disclosure is shown;

[0011] Figure 6 An example image detection and recognition method according to one aspect of the present disclosure is shown;

[0012] Figure 7 An example method of creating and retrieving an individual's offline track according to one aspect of the present disclosure is shown; and

[0013] Figure 8 An example system including various hardware computing components according to an aspect of the present disclosure is shown.

[0014] Example Embodiments

[0015] Various example embodiments of the present disclosure are discussed in detail below. Although specific implementations are discussed, it should be understood that this is for illustrative purposes only. Those skilled in the relevant art will recognize that other components and configurations may be used without departing from the spirit and scope of the present disclosure. Therefore, the following description and drawings are illustrative and should not be interpreted as limiting. Many specific details are described to provide a thorough understanding of the present disclosure. However, in some instances, in order to avoid making the description difficult to understand, well-known or conventional details are not described. References to an embodiment or embodiments in the present disclosure may be references to the same embodiment or any embodiment; and, these references represent at least one embodiment.

[0016] Reference to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The phrase "in one embodiment" appearing in various places in the specification is not necessarily all referring to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive of other embodiments. In addition, various features are described that may be exhibited by some embodiments and not by other embodiments.

[0017] The terms used in this specification generally have their ordinary meaning in the art in the context of the present disclosure and in the specific context in which each term is used. Alternative language and synonyms may be used for any one or more terms discussed herein, and no matter whether the terms are elaborated or discussed in this article, there is no special meaning. In some cases, synonyms for certain terms are provided. The description of one or more synonyms does not exclude the use of other synonyms. The use of examples (including examples of any terms discussed herein) anywhere in this specification is only illustrative and is not intended to further limit the scope and meaning of the present disclosure or any example terms. Likewise, the present disclosure is not limited to the various embodiments provided in this specification.

[0018] Without intending to limit the scope of the present disclosure, examples of instruments, devices, methods and related results thereof according to embodiments of the present disclosure are given below. Note that, for the convenience of the reader, titles or subtitles may be used in the examples, which should never limit the scope of the present disclosure. Unless otherwise specified, the technical and scientific terms used herein have the meanings commonly understood by those of ordinary skill in the art to which the present disclosure belongs. In the event of a conflict, this document (including definitions) shall prevail.

[0019] Additional features and advantages of the present disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the principles disclosed herein. The features and advantages of the present disclosure may be realized and obtained by means of the instruments and combinations particularly noted in the appended claims. These and other features of the present disclosure will become more apparent from the following description and the appended claims or may be learned by practicing the principles set forth herein.

[0020] An execution endpoint (EE) may include a system with computing capabilities that can run a function. Non-limiting examples may include computers, laptops, IoT devices, servers, switches, mobile phones, kiosks, workstations, etc. An EE may be registered in a function router for executing functions. An execution endpoint may run various FaaS runtime environments and services.

[0021] The client may include a device and / or application that attempts to execute a function on an execution endpoint. Non-limiting examples of the client may include a robotic arm, a mobile phone, a handheld scanner, an application, a printer, a kiosk, etc.

[0022] Functions can include snippets of code. A snippet of code can represent, for example, a short-lived, self-contained set of business logic. Serverless functions can be compared to stored procedures in that they perform a specific action and are called and executed when needed, returning to sleep (but ready) only when execution is complete.

[0023] A location may include a physical location (e.g., a building, a floor, etc.) and / or a logical location. A location may be associated with a specific latitude and longitude coordinate. For example, a location may refer to a specific latitude and longitude coordinate corresponding to a manufacturing floor where a robot is located, or a conference room where a FaaS device is plugged in, or an area associated with an environment.

[0024] Function routing rules can include policies and controls about who, what, when, where, why, and / or how functions are executed. These rules can include IT-defined guardrails that affect the entire system, as well as other rules specified by IT or development teams for specific functions. Example rules can include: function A can run on any endpoint, but function B must only run on private endpoints; or function A can be called by any client at a specific location, but function B can only be called by a specific client at any location.

[0025] Overview

[0026] Systems, methods, and computer-readable media are disclosed for a hybrid cloud architecture that provides machine learning-based object recognition to create offline tracks of users and consumers.

[0027] In one aspect of the present disclosure, a system includes: one or more access points with video capabilities; and one or more processors configured to receive image data from the one or more access points with video capabilities; execute a first process at a first processor in the one or more processors to detect one or more objects of interest in the image data; generate a vector ID for the one or more objects detected in the image data; execute a second process at a second processor in the one or more processors to identify the one or more objects in the vector ID; and generate at least one offline track for the one or more objects based on statistical information associated with the identified one or more objects.

[0028] In one aspect, a method includes: receiving image data from one or more video-capable access points; performing a first process at a first processor to detect one or more objects of interest in the image data; generating vector IDs for the one or more objects detected in the image data; sending the vector IDs to a second processor; performing a second process at the second processor to identify the one or more objects in the vector IDs; and generating offline tracks for the one or more objects based on statistical information associated with the identified one or more objects.

[0029] In one aspect, one or more non-transitory computer-readable media having computer-readable instructions that, when executed by one or more processors, cause the one or more processors to: receive image data from one or more video-capable access points; perform a detection process at a first processor to detect one or more objects of interest in the image data; generate vector IDs for the one or more objects detected in the image data; send the vector IDs to a second processor; perform a recognition process at the second processor to identify the one or more objects in the vector IDs; and generate offline tracks for the one or more objects based on statistical information associated with the identified one or more objects. DETAILED DESCRIPTION

[0030] The disclosed technology addresses the need in the art for an efficient image recognition process that can be used to track the offline trajectory of an object or individual as the object or individual moves through one or more physical locations and / or portions thereof.

[0031] The disclosure begins with a description of an example network architecture. Figure 1 An example hybrid cloud architecture for image recognition according to one aspect of the present disclosure is shown.

[0032] Figure 1 An example hybrid cloud system 100 is shown that includes three layers. The layers include a fog layer 102, a private platform layer 104, and a public platform layer 106.

[0033] The fog layer 102 may include two sub-layers 108 and 110. Sub-layer 108 may include one or more access points, for example, access point (AP) 112. Access point 112 is capable of capturing video and image data of objects and individuals (e.g., individual 113) in its field of view (FOV), and may provide camera services. Although individual 113 is used as an example below, the inventive concept is not limited to individuals and persons, but may also include any other type of moving objects, animals, etc. In one example, each individual 113 may have a shopping and / or mobile device associated with it.

[0034] In another example, a video-capable access point 112 may be coupled to a mobile camera that may be attached, for example, to a shopping cart carried by an individual 113 when the individual 113 visits a commercial site or a particular location. This may provide various types of metadata that may be associated with the individual 113 (e.g., types of products selected for purchase, types of products viewed but not selected, etc.), which may convey an improved and more accurate offline track of the individual 113.

[0035] When capturing image / video data, more than one object / individual may be present in the FOV of the access point 112. The access point 112 may be any known or to-be-developed IP-based video / image capture device, such as access points with video / image capture capabilities from Cisco Systems, Inc. of San Jose, Calif. These access points with video capabilities may be referred to hereinafter simply as access points.

[0036] Access points 112 may be installed throughout physical locations or venues (e.g., department stores, shopping malls, stadiums, libraries, college campuses, etc.) that may continuously (or at predetermined time intervals) capture image and video data of objects within their FOV.

[0037] Sublayer 110 may be a fog / edge service that includes one or more servers (which may be any known or to be developed server, such as Cisco Technologies, Inc. of San Jose, California) that may perform a detection process (a facial detection process) on video and image data captured by access points 112 in sublayer 108. A non-limiting example of such a fog server is a Raspberry Pi device / server. Alternatively, the detection process performed by the fog server may be performed at private platform layer 104, as described below. Reference will be made to Figure 2 Describes an example structure and components of a fog server.

[0038] The private platform layer 104 may be a private cloud including one or more servers providing container-based services, and may be located geographically near or in the premises where the components of the fog layer 102 are installed (e.g., within the same confined physical space / building, within a threshold physical distance of, for example, a few hundred feet or a few miles), or in the alternative may be remote from but communicatively coupled thereto. The private platform layer 104 may be a platform developed by Cisco Technologies, Inc. of San Jose, California, and may therefore be referred to as Cisco Container Platform (CCP) 104. The private platform layer 104 may be an infrastructure as a service (IaaS) having one or more servers (allowing automatic deployment, scaling, and management of container-based applications), one or more storage facilities, etc., as described below. The one or more servers of the private platform layer 104 may be one or more server nodes (e.g., K8 master nodes) that may be managed by other server nodes (e.g., Kubemetes (K8) nodes). The private platform layer 104 may also include any other components necessary for its operation, including but not limited to image processing and facial detection algorithms, components for establishing communications with other components of the system 100 (e.g., components in the fog layer 102 or the public platform layer 106, etc.). Figure 2 An example structure and components of application-specific platform layer 104 are described.

[0039] The public platform layer 106 may be a third-party cloud platform provided by a third party, and thus may be referred to as an external cloud platform (ECP) 106. The public platform layer 106 may be a public, private, or hybrid cloud platform. Alternatively, the public platform layer 106 may be provided by the same entity that provides the private platform layer 104 (e.g., Cisco Technologies, Inc. of San Jose, California).

[0040] There may be a secure communication channel between the private platform layer 104 and the public platform layer 106, through which various services provided by the components / services of the private platform layer 104 can call services and functions provided at the public platform layer 106. In addition, offline tracks of individuals and objects can be accessed by logging into the system 100 via a terminal 114, as described below. The terminal 114 can be any known or to be developed computer device capable of establishing a connection to a server at various levels of the system 100, including but not limited to mobile devices, laptops, desktop computers, tablet computers, etc. The terminal 114 can be communicatively coupled to components at the public platform layer 106, the private platform layer 104, and / or the fog layer 102.

[0041] Figure 2 According to one aspect of the present disclosure Figure 1 Components of the fog server at the fog layer.

[0042] The fog server 200 may be a physical or virtual network node (e.g., a kubelet node) having one or more memories (e.g., memory 202) and one or more processors (e.g., processor 204) (which together may provide a hosting / interfacing environment). The memory 202 may have one or more services installed thereon that perform their respective functions when executed by the processor 204. For example, the memory 202 may include a camera / sensor service 206, an object detection service 208, a stream service 210, and a coordinator service 212.

[0043] The camera / sensor service 206, when executed by the processor 204, can manage, control, and update the sensors and access points 112. The fog server 200 is communicatively coupled to each access point 112 and the associated image capture cameras and sensors, and can continuously receive captured image and video data from the access points 112.

[0044] Object detection service 208, when executed by processor 204, may perform any known or to be developed operations on the received image and video data to detect one or more objects of interest in the received data, as described below. Information extracted by processor 204 from the received video and image data may include facial information of individual 113, the location of objects included in the captured image of individual 113, various metadata associated with individual 113, including but not limited to the time of day, GPS location of the venue, products / items selected by individual 113, activities engaged in by individual 113 in the venue where access point 112 is installed, etc. The extracted information may then be sent as a digital vector to a server at private platform layer 104 for further processing, as described below.

[0045] The flow service 210 manages the operation and communication of network nodes within the system 100 when executed by the processor 204. For example, the flow service 210 may be a Kubelet flow service that manages the interoperability of kubelet nodes, as is known in the art.

[0046] The coordinator service 212, when executed by the processor 204, can manage and coordinate the exchange of information between different layers of the hybrid architecture of the system 100 for performing different image processing and recognition functions, as described below.

[0047] Figure 3 According to one aspect of the present disclosure Figure 1 Components of the server at the private platform layer.

[0048] The server 300 may be a physical or virtual network node (e.g., a kubelet node) having one or more memories (e.g., memory 302), one or more processors (e.g., processor 304), and one or more databases 305. In the case of a virtual network node, the server 300 may be implemented on a HyperFlex hyper-converged system, which may combine software-defined storage devices and data service software with a unified computing system to provide a converged infrastructure system that integrates computing, networking, and storage resources to improve efficiency and enable centralized management of the system 100.

[0049] The memory 302 may have one or more services installed thereon that perform their respective functions when executed by the processor 304. For example, the memory 302 may include a profile service 306, an activity / tag service 308, a vector search service 310, a machine learning training service 312, and a coordinator service 314.

[0050] The activity / tagging service 308, when executed by the processor 304, may associate vector IDs (received from the server 200 after performing a facial detection process on video or image data captured by the access point 112) as having a threshold amount of similarity therebetween, without associating the vectors with a given user ID. For example, if the recognition process is performed on the same face of an individual 113 with slight variations (e.g., variations in angle, facial expression, glasses, etc.), the tagging service 308 may be implemented to determine that two different vectors are close enough to both belong to the same person. This process may be referred to as user ID-independent vector correlation, which is a novel advantage of the hybrid architecture of the system 100. Because these vectors are sent to the server 300 "anonymously" and tagged as similar, unauthorized access to these vectors cannot reveal the true identity of the user associated with these vectors, thereby protecting the privacy and security of the identities of the respective users.

[0051] The profile service 306, when executed by the processor 304, may establish a link or association of such a vector ID with a given user ID. For example, the individual 113 may have a profile established with the system 100 that also includes their picture (picture ID), or the individual 113 may log into the system 100 via the terminal 114. An image of the individual 113 taken at the time of login, or an image that may have been stored in association with the profile of the individual 113, may be used to associate the relevant vector ID with the user ID, and the result is stored in the database 305.

[0052] The vector search service 310, when executed by the processor 304, may perform any known or to be developed image recognition processing of digital vectors representing detected images received after the image detection process performed by the fog server 200. Vector search for image recognition is further described below.

[0053] The machine learning training service 312 may be a classifier that continuously receives various data related to objects and individuals from the terminals 114 or the common platform layer 106 and develops a machine learning-based model that is used by the vector search service to identify images of interest in the data received from the fog server(s) 200. The training of such a classifier may be based on any known or to-be-developed machine learning process.

[0054] In another example, machine learning training service 312 can generate new data (e.g., multiple different poses of an individual from a single photo of the same individual received via endpoint 114 or public platform layer 106, as described below).

[0055] The coordinator service 314, when executed by the processor 304, can manage and coordinate information exchange between different layers of the hybrid structure of the system 100 to perform different image processing and recognition functions, as described below. The coordinator service 314 can communicate or synchronize with the coordinator service 212 of the fog server 200.

[0056] Figure 4 According to one aspect of the present disclosure Figure 1 Components of the server at the common platform layer.

[0057] The server 400 may be a physical or virtual network node (eg, a kubelet node) having one or more memories (eg, memory 402 ), one or more processors (eg, processor 404 ), and one or more databases 405 .

[0058] The memory 402 may have one or more services installed thereon that perform their respective functions when executed by the processor 404. For example, the memory 402 may include a coordinator service 406 and a machine learning training service 408.

[0059] The coordinator service 406, when executed by the processor 404, can manage and coordinate the information exchange between different layers of the hybrid structure of the system 100 to perform different image processing and recognition functions, as described below. The coordinator service 406 can communicate or synchronize with the coordinator service 314 of the server 300 and / or the coordinator service 212 of the fog server 200.

[0060] The machine learning training service 408 may be the same as the machine learning training service 312 of the server 300 described above, or may work in conjunction with the machine learning training service 312 to train a model set for the server 300 that will be used when performing vector searches on data vectors received from the fog server 200 to detect faces and perform image recognition.

[0061] The database 405 may be one or more public, private, and / or hybrid databases that collect and store information about users and objects. For example, the database 405 may include various different images of individuals available on the Internet, various social media websites, etc. via the public cloud layer 106 (e.g., Google database, Facebook, which may be referred to as a public cloud platform). The machine learning training service 408 may retrieve these different images and associate them or use them to train a model set on the server 300, which may be used to better and more accurately detect user images.

[0062] Having described various components of an example hybrid cloud architecture for image detection and recognition, an example method implemented by such a hybrid cloud architecture will be described below.

[0063] Figure 5 An example method for developing a machine learning-based model for object recognition according to one aspect of the present disclosure is shown. Figure 5 However, it will be understood that one or more processors, such as processor 304 of server 300, implement one or more services stored on memory 302 of server 300 to perform the reference Figure 5 Describes the functionality.

[0064] At S500, the server 300 may receive data from the common platform layer 106 or its server 400. The received data may include various types of information about users, their associated identities, activities and information, captured images, etc. Alternatively, at S500, the server 300 may query the server 400 for such information. The query may be in response to execution of the vector search service 310 performed by the processor 304. In another example, the data received at S500 may be the output of the machine learning training service 408 in the form of a data set, which may be stored by the server 300 in the database 405 for vector-based facial recognition, as described below.

[0065] At S502 (which may be at the same time, before or after step S500), the server 300 may receive a user registration request. Such a request may be received via a user interface (UI) on the terminal 114. The user registration request may be a request made by a user using the terminal 114. The user may provide or register a corresponding user identification (ID) request. Together with such a request, the user may also provide a photo ID.

[0066] At S504 and after receiving the user ID and the photo ID, the server 300 may generate multiple different versions of the photo ID (which may also be referred to as a reference image). Each such version may be generated by, for example, rotating the photo ID by a specific angle (e.g., rotating by 15 degrees), applying Gaussian noise to the photo ID or each rotated version thereof, and / or placing the photo ID horizontally or vertically. Thus, multiple different versions of the user's photo ID may be generated, which may in turn be used to better train the model set developed by executing the machine learning training service 312.

[0067] At S506, the server 300 may perform a machine learning training service 312 on the data received at S500 and / or the photo IDs received and generated at S502 and S504. In one example, the machine learning training service may extract various features and characteristics of the received data and photo IDs, associate them with each other and store them in a database such as database 305. In one example, each user ID may have a specified photo ID and multiple other photos associated with it. The multiple additional photos may be referred to as life photos to indicate that the additional photos are collected and provided over time at different instances and belong to the same user.

[0068] At S508 and based on the machine learning training of S506, the server 300 generates a model set. The extracted features together with the corresponding user identification information can be stored as a digital data set for vector-based search for facial recognition, as described below.

[0069] Therefore, by implementing Figure 5 The process, continuously or at set intervals, the server 300 develops a method for Figure 6 A model set of processes to receive face detection results from the fog server 200 and perform an efficient and secure process for image recognition to identify objects and users.

[0070] Figure 6 An example image detection and recognition method according to one aspect of the present disclosure is shown. The following describes the image detection and recognition method from the perspective of fog server 200 and server 300. Figure 6However, it will be understood that one or more processors, such as the processor 204 of the fog server 200 and the processor 304 of the server 300, implement one or more services stored on a corresponding one of the memories 202 and 302 to perform the reference Figure 6 Describes the functionality.

[0071] At S600, images and / or video data of an object or individual (eg, individual 113) may be captured by one or more access points, such as one or more access points 112, as described above. In one example, the capturing of images and / or video data may be performed continuously.

[0072] At S602, the fog server 200 performs a detection process on the image and / or video data captured by (one or more) access points 112 at S600. In one example, the server 200 may perform the detection process using a modified version of a multi-task cascaded convolutional neural network (MTCNN) to detect an object of interest (e.g., a face). As is known, the MTCNN has three outputs, namely, facial classification confidence, facial location, and facial landmarks. According to the modified version, a fourth output is added that indicates clarity information of the detected face in order to distinguish between blurred faces and clear faces detected in the image. In one example, classifying an image or a detected face as a blurred image or a clear image is a classification problem, which can be solved using a cross entropy loss according to the formula provided below:

[0073]

[0074] Where i represents the i-th example in the training dataset, and y i ∈{0, 1} represents the ground truth label, and p i is the probability that the neural network represents the sharpness of the detected face.

[0075] At S604, the server 200 sends the detection result as a digital vector to the server 300. Thus, when multiple cameras or access points 112 capture multiple users or different images of the same user, each detected instance is sent to the server 300 as an independent digital vector without being associated with a specific user ID, which is a process performed by the server 300 as described below.

[0076] At S606, the server 300 performs an image recognition process on the digital vector received from the server 200 to identify the user. Figure 5The vector search service 310 is performed using a model set developed by the process of performing vector search service 310, as described above. Thus, (one or more) vector IDs are associated with user IDs or photo IDs stored in database 305. An example algorithm for such a vector search is an approximate nearest neighbor search, which involves determining the Euclidean distance between two vectors to determine the similarity between them.

[0077] At S608, the generated vector ID(s) (recognition results) are stored in a database (eg, database 305) in association with the corresponding user ID(s), together with metadata thereof corresponding to various statistical information.

[0078] As more and more images of individuals 113 are based on Figure 5 and Figure 6 By being captured, processed, identified and stored in the process, an offline trajectory of the individual can be created, which can be used to study and track the habits of individual 111 (shopping habits, visited places, more detailed details of individual 113's movements within the visited places, etc.).

[0079] Figure 7 An example method of creating and retrieving an individual's offline track according to one aspect of the present disclosure is shown. Figure 7 However, it will be understood that one or more processors, such as processor 304 of server 300, implement one or more services stored on memory 302 of server 300 to perform the reference Figure 7 Describes the functionality.

[0080] At S700, a request for a user's offline track is received. The request may be a request from an individual, such as individual 113 as described above, who wants to view his or her own offline track (e.g., via terminal 114). Alternatively, the request may come from a network operator, or an operator of a location where system 100 is used (where access point 112 and / or server(s) 200 and server(s) 300 are installed).

[0081] At S702, image data of the user for which a user offline track request is received is obtained (where the requester is individual 113). The image may be a previously recorded profile image or a newly provided image.

[0082] At S704, the image data of S702 is associated / matched with a detected previous image of the same user instance (stored as a vector ID in the (one or more) databases 305). In this process, a vector ID is first generated for the image data of S702 by performing a detection and recognition process in a manner similar to that described above. Thereafter, a vector-based search (e.g., using an index) is performed to find a match (matching vector ID) in the stored vector IDs for the vector ID associated with the image data of S702.

[0083] Thereafter, at S706 , an offline trajectory is generated. In this process, statistical information associated with the matching vector ID is retrieved (eg, from the database 305 ) and compiled into a report, the format of which may be a format requested by the requesting party, a table form, etc.

[0084] At S708 , the offline trajectory is sent (transmitted) to the requesting party.

[0085] In one example, the requesting party from which the request is received at S700 may not be the individual himself, but may be an operator or manager of one or more venues utilizing the system 100. Thus, instead of performing S702 and S704 as described above, various levels of statistical information regarding patrons and individuals detected at the underlying venues may be collected and retrieved, as described above.

[0086] Having described various examples of performing facial detection and recognition using components and methods implemented by components of a hybrid cloud architecture, the example system components may be implemented as one or more of servers 200 , 300 , and / or 400 .

[0087] Figure 8 An example system including various hardware computing components according to one aspect of the present disclosure is shown. When practicing the present technology, more appropriate embodiments will be obvious to those of ordinary skill in the art. Those of ordinary skill in the art will also readily appreciate that other system embodiments are also possible.

[0088] Figure 8A system architecture 800 is shown in which the components of the system are in electrical communication with each other using connections 806. The exemplary system 800 includes a processing unit (CPU or processor) 804, and a system connection 806 that couples various system components including a system memory 820 such as a read-only memory (ROM) 818 and a random access memory (RAM) 816 to the processor 804. The system 800 may include a cache of high-speed memory that is directly connected to the processor 804, is adjacent to the processor 804, or is integrated as part of the processor 804. The system 800 can copy data from the memory 820 and / or storage device 808 to the cache 802 for quick access by the processor 804. In this way, the cache can provide a performance boost and avoid delays in the processor 804 while waiting for data. These modules and other modules can be controlled or configured to control the processor 804 to perform various actions. Other system memories 820 can also be used. The memory 820 can include a variety of different types of memory with different performance characteristics. Processor 804 may include any general purpose processors, such as service 1 810, service 2 812, and service 3 814 stored in storage device 808, configured to control processor 804 as well as dedicated processors, where software instructions are incorporated into the actual processor design. Processor 804 may essentially be a fully self-contained computing system, including multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors may be symmetric or asymmetric.

[0089] In order to enable the user to interact with the system 800, the input device 822 can represent any number of input mechanisms, for example, a microphone for voice, a touch-sensitive screen for gesture or graphic input, a keyboard, a mouse, motion input, voice, etc. The output device 824 can also be one or more of many output mechanisms known to those skilled in the art. In some cases, a multimodal system can enable the user to provide multiple types of input to communicate with the system 800. The communication interface 826 can generally dominate and manage user input and system output. There is no restriction on the operation on any specific hardware arrangement, so when developing, the basic functions of this article can be easily replaced with improved hardware or firmware arrangements.

[0090] The storage device 808 is a non-volatile memory and may be a hard disk or other type of computer-readable medium that can store data accessible by a computer, such as a magnetic tape, a flash memory card, a solid-state storage device, a digital versatile disk, a cartridge, a random access memory (RAM) 816, a read-only memory (ROM) 818, and a mixture thereof.

[0091] System 800 may include an integrated circuit 828 , such as an application specific integrated circuit (ASIC) configured to perform various operations. Integrated circuit 828 may be coupled to connection 806 to communicate with other components in system 800 .

[0092] The storage device 808 may include software services 810, 812, 814 for controlling the processor 804. Other hardware or software modules are contemplated. The storage device 808 may be connected to the system connection 806. In one aspect, a hardware module that performs a particular function may include a software component stored in a computer-readable medium in association with the necessary hardware components (e.g., the processor 804, the connection 806, the output device 824, etc.) to perform the function.

[0093] For clarity of explanation, in some cases, the present technology may be presented as including separate functional blocks including functional blocks having devices, device components, steps or routines in methods embodied in software, or a combination of hardware and software.

[0094] In some embodiments, computer-readable storage devices, media, and memories may include cables or wireless signals containing bit streams, etc. However, when referred to, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0095] The method according to the above example can be implemented using computer executable instructions stored in or obtainable from a computer readable medium. Such instructions may include, for example, instructions and data that enable or configure a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or group of functions. The used portion of the computer resources may be accessed via a network. The computer executable instructions may be, for example, binary intermediate format instructions, such as assembly language, firmware, or source code. Examples of computer readable media that can be used to store instructions, information used, and / or information created during the method according to the example include disks or optical disks, flash memory, USB devices equipped with non-volatile memory, network storage devices, etc.

[0096] Devices implementing methods according to these disclosures may include hardware, firmware, and / or software, and may take any of a variety of form factors. Typical examples of such form factors include laptop computers, smart phones, small form factor personal computers, personal digital assistants, rack-mounted devices, stand-alone devices, etc. The functionality described herein may also be embodied in peripheral devices or add-on cards. As a further example, such functionality may also be implemented on circuit boards between different chips or different processes executed in a single device.

[0097] Instructions, media for communicating such instructions, computing resources for executing them, and other structures for supporting such computing resources are means for providing the functions described in these disclosures.

[0098] Although various examples and other information are used to explain various aspects within the scope of the appended claims, limitations on the claims should not be implied based on specific features or arrangements in such examples, as those of ordinary skill in the art will be able to use these examples to derive various implementations. In addition, although some topics may have been described in language specific to examples of structural features and / or method steps, it should be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or actions. For example, such functionality may be distributed or performed differently in components other than those identified herein. In addition, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims.

[0099] Claim language that recites "at least one of a group" means that one member of the group or multiple members of the group satisfy the claim. For example, claim language that recites "at least one of A and B" means A, B, or A and B.

Claims

1. A system for object recognition, comprising: one or more video-capable access points; as well as One or more processors configured to: receiving image data from the one or more video-capable access points; detecting, via a first processor of the one or more processors, one or more objects of interest in the image data, the first processor being located at a first layer of a hybrid cloud system; sending a plurality of digital vectors to a second processor of the one or more processors, each of the plurality of digital vectors representing a respective object of the one or more detected objects; identifying, via a second processor of the one or more processors, the one or more objects using the plurality of digital vectors, wherein the second processor is located at a second layer of the hybrid cloud system, and wherein the second processor implements a model trained to identify objects by digital vectors, the model being trained at least in part using a plurality of images generated from images of the objects, depicting the objects in different poses in the images; and At least one offline track is generated for the one or more objects based on statistical information associated with the identified one or more objects.

2. The system according to claim 1, wherein: The one or more objects are individuals.

3. The system according to claim 1 or 2, wherein: The first layer is a fog layer of a hybrid cloud architecture associated with the hybrid cloud system.

4. The system according to claim 3, wherein: The second processor is physically located near the first processor and the one or more video-capable access points and forms a private platform layer of the hybrid cloud architecture.

5. The system according to claim 1 or 2, wherein: The first processor is communicatively coupled to the second processor.

6. The system according to claim 1 or 2, wherein: The model is trained to perform a vector-based search on a database for identifying the one or more objects.

7. The system according to claim 6, wherein: The model includes a model based on machine learning.

8. The system according to claim 7, wherein: The second processor is configured to create the machine learning-based dataset using image data retrieved from a public cloud platform and one or more computer-generated versions of reference images of the one or more objects.

9. The system according to claim 8, wherein: The second processor is configured to: receiving the reference image; and At least one of rotating or flipping the reference image is performed to generate a corresponding computer-generated version of the reference image.

10. A method for object recognition, comprising: receiving image data from one or more video-capable access points; detecting, via a first processor of the one or more processors, one or more objects of interest in the image data, the first processor being located at a first layer of a hybrid cloud system; sending a plurality of digital vectors to a second processor of the one or more processors, each of the plurality of digital vectors representing a respective object of the one or more detected objects; identifying, via the second processor, the one or more objects using the plurality of digital vectors, wherein the second processor is located at a second layer of the hybrid cloud system, and wherein the second processor implements a model trained to identify objects by digital vectors, the model being trained at least in part using a plurality of images generated from images of the objects, depicting the objects in different poses in the images; and An offline track is generated for the one or more objects based on statistical information associated with the identified one or more objects.

11. The method according to claim 10, wherein: The one or more objects include at least one individual.

12. The method according to claim 10 or 11, further comprising: A request for the offline track is received from at least one requestor.

13. The method according to claim 12, wherein: The offline track indicates the history of the individual's visiting locations and shopping habits.

14. The method according to claim 10 or 11, wherein: The first processor is communicatively coupled to the second processor.

15. The method according to claim 10 or 11, wherein: The model is trained to perform a vector-based search on a database for identifying the one or more objects.

16. The method according to claim 15, wherein: The model includes a machine learning based model and the vector-based search uses a machine learning based dataset.

17. The method according to claim 16, further comprising: The machine learning-based model is created using image data retrieved from a public cloud platform and one or more computer-generated versions of reference images of the one or more objects.

18. The method according to claim 17, further comprising: receiving the reference image; and A corresponding computer-generated version of the reference image is generated by rotating or flipping at least one of the reference images.

19. One or more non-transitory computer-readable media comprising computer-readable instructions which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 10 to 18.

Images