Reconnaissance data filtering techniques

Abstract

A system for identifying desired information from collected sensor data includes a collection device and a processing module. The collection device collects the sensor data, coarsely filters the sensor data according to predefined rules, generates filter-matched data, and securely transmits the filter-matched data to the processing module. The processing module finely filters the filter-matched data, generates desired information, provides the desired information to an authorized actor, and deletes the filter-matched data.

Description

[[TECHNICAL FIELD]]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 202,954 (Attorney Docket No. AA610), filed July 1, 2021 and entitled "SURVEILLANCE DATA FILTRATION TECHNIQUES", which is incorporated herein by reference in its entirety.

[0002] The present disclosure generally relates to electronic search or reconnaissance. More specifically, the present disclosure relates to identifying locations of targets for law enforcement agencies and filtering collected data associated therewith. [[BACKGROUND ART]]

[0003] Autonomous vehicles associated with law enforcement agencies can, for example, determine whether a vehicle violates road traffic laws, pursue the vehicle, and electronically issue tickets or warnings to violators. Autonomous vehicles can be trained to find good hiding spots to catch traffic violators, point their cameras to accurately monitor traffic, identify vehicles, analyze incoming data against a database of road traffic laws, and communicate with a central computing system and reconnaissance cameras. Such autonomous vehicles can record collected data and transmit the records to government agencies. Autonomous vehicles can also assess the environment and people therein to reduce risks to law enforcement personnel. For example, drones can observe and interact with suspects, capture images of suspects and associated documents of interest, perform image comparison, perform text extraction and classification, correlate text with image identification, and communicate data to a base station.

[0004] The transmitted images can be encrypted, for example, to address privacy concerns. For instance, privacy issues associated with the collection of image datasets, such as reconnaissance or medical data, may be addressed using a learnable encryption algorithm. It is also possible to protect people's privacy to a high degree while taking photographs and recording them to investigate incidents such as crimes and acts of terrorism. For example, cameras may be installed in private vehicles, and images from such cameras may then be stored for short time periods, such as one to two weeks, in case they are needed by law enforcement agencies.

[0005] A common feature of many such systems is that they provide data to law enforcement agencies, and therefore all data collected relates to suspects about whom law enforcement has the right to collect data. However, some of these systems, such as autonomous traffic monitoring and environmental assessment, may be involved in collecting a range of data that does not relate to any person or thing about whom law enforcement has the right to collect data. Systems that can identify the location of objects and items of interest may not be able to do so while also protecting the privacy of objects and items that are not of interest.

[0006] The background described above is intended merely to provide a contextual overview of some existing issues and is not intended to be exhaustive. [Overview of the project] [Means for solving the problem]

[0007] In the following description, numerous specific details are provided to offer a thorough understanding of the various aspects and arrangements. However, those skilled in the art will recognize that the techniques described herein may be practiced without one or more of the specific details, or using other methods, components, materials, etc. In other cases, well-known structures, materials, or operations may not be shown or described in detail to avoid obscuring certain aspects.

[0008] Throughout this specification, references to “an aspect,” “an arrangement,” or “a configuration” indicate that a particular feature, structure, or characteristic is being described. Therefore, throughout this specification, expressions such as “in one aspect,” “in one arrangement,” “in a configuration,” or equivalent phrases in various places do not necessarily refer to the same aspect, feature, structure, or arrangement. Furthermore, particular features, structures, and / or characteristics may be combined in any preferred manner.

[0009] To the extent used in this disclosure and claims, the terms “component,” “system,” “platform,” “layer,” “selector,” “interface,” and their equivalents are intended to refer to computer-related entities or entities relating to operable devices with one or more specific functionalities, which may be hardware, a combination of hardware and software, software, or running software. For illustrative purposes, a component may be, but is not limited to, a process launched on a processor, a processor, an object, an executable file, a thread of execution, a program, and / or a computer. For illustrative purposes, but is not limited to, an application launched on a server and the server itself may be components. One or more components may reside within a process and / or a thread of execution, and components may be localized on one computer and / or distributed across two or more computers. In addition, components may run from various computer-readable media, device-readable storage devices, or machine-readable media having various data structures stored thereon. Components may communicate via local and / or remote processes, such as according to signals, which have one or more data packets (e.g., data from a local system, another component in a distributed system, and / or data from one component interacting with other systems via signals across a network such as the Internet). In another embodiment, a component may be a device with specific functionality provided by mechanical parts operated by an electrical or electronic network, which may be operated by a software or firmware application run by a processor, which may reside inside or outside the device and execute at least part of the software or firmware application.In another embodiment, the component may be a device that provides specific functionality through electronic components without involving mechanical parts, and the electronic components may include a processor, at least in part, to run software or firmware that gives the electronic components their functionality.

[0010] To the extent used herein, terms such as “store,” “storage,” “data store,” “data storage,” “database,” and their equivalents refer to memory components, entities embodied in memory, or components containing memory. It will be understood that the memory components described herein may be either volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0011] In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or evident from the context, “X adopts A or B” is intended to mean any of the natural inclusive enumerations. That is, if X adopts A, X adopts B, or X adopts both A and B, “X adopts A or B” is satisfied under any of the aforementioned cases. Furthermore, the articles “a” and “an” as used in this disclosure and claims should generally be interpreted as “one or more” unless otherwise specified or evident from the context that they refer to a singular form.

[0012] The words “exemplary” and / or “demonstrative” mean, to the extent used herein, an example, case, or illustration. To avoid doubt, the subject matter disclosed herein is not limited by the disclosed examples. In addition, any aspect or design described herein as “exemplary” and / or “demonstrative” should not necessarily be construed as being preferable or more advantageous than other aspects or designs, nor should it be meant to exclude equivalent exemplary structures and techniques known to those skilled in the art. Furthermore, to the extent that “includes,” “has,” “contrains,” and other similar words are used in either the detailed description or a claim, such terms are intended to be comprehensive in a manner similar to the non-restrictive transitional term “comprising,” without excluding any additional or other elements.

[0013] As used herein, the terms “infer” or “inference” generally refer to the process of inferring about or inferring the state of a system, environment, user, and / or intent from a set of observations such as those captured through events and / or data. Captured data and events may include user data, device data, environmental data, data from sensors, application data, implicit data, explicit data, etc. Inference can be employed, for example, to identify a specific context or action, or to generate a probability distribution across states of interest based on an examination of the data and events.

[0014] The disclosed subject matter may be implemented as a method, apparatus, or article using standard programming and / or engineering techniques to produce software, firmware, hardware, or any combination thereof, and to control a computer to implement the disclosed subject matter. The term “article of manufacture” is intended, to the extent used herein, to encompass any computer-readable device, machine-readable device, computer-readable transport device, computer-readable medium, or computer program accessible from a machine-readable medium. For example, computer-readable medium includes, but is not limited to, magnetic storage devices such as hard disks, floppy disks, magnetic stripes, and optical discs (e.g., compact discs (CDs), digital video discs (DVDs), and Blu-ray discs). TM This may include (BD), smart cards, flash memory devices (e.g., cards, sticks, key drives), virtual devices that emulate storage devices, and / or any combination of the above computer-readable media.

[0015] Generally, program modules include routines, programs, components, data structures, etc., that perform a specific task or implement a specific abstract data type. Illustrated embodiments of this disclosure may be practiced in a distributed computing environment where a task is performed by remote processing devices linked through a communication network. In a distributed computing environment, program modules may reside in both local and remote memory storage devices.

[0016] A computing device may include at least computer-readable storage media, machine-readable storage media, and / or communication media. Computer-readable or machine-readable storage media can be any available storage media that can be accessed by a computer, and include both volatile and non-volatile media, and removable and non-removable media. In some embodiments, but not limited to them, computer-readable or machine-readable storage media may be implemented in any way or technique for storing information such as computer-readable or machine-readable instructions, program modules, structured data, or unstructured data.

[0017] Computer-readable storage media can include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc (BD), or other optical disc storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices, or other magnetic storage devices, solid-state drives or other solid-state storage devices, or other tangible and / or non-transitory media that can be used to store desired information. In this regard, it should be understood that the terms “tangible” or “non-transitory” used herein as applied to storage devices, memory, or computer-readable media are modifiers that exclude only the propagating transient signal itself, and not exclude any standard storage device, memory, or computer-readable media that are not limited to the propagating transient signal itself.

[0018] Computer-readable storage media can be accessed by one or more local or remote computing devices for various operations on the information stored in the media, for example, through access requests, queries, or other data retrieval protocols.

[0019] The system bus may be one of several types of bus structures, which may be further interconnected to a memory bus (with or without a memory controller), peripheral bus, and local bus using any of various commercially available bus architectures, as may be used herein. The database may include a basic input / output system (BIOS), which may be stored in non-volatile memory such as ROM, EPROM, or EEPROM, as may be used herein, and which contains basic routines that help transfer information between elements within the computer, such as during startup. RAM may also include high-speed RAM, such as static RAM, for caching data.

[0020] As used herein, a computer may operate within a network environment using logical connectivity via wired and / or wireless communication to one or more remote computers. Remote computers may be workstations, servers, routers, personal computers, portable computers, microprocessor-based entertainment appliances, peer devices, or other common network nodes. Logical connectivity as described herein may include wired / wireless connectivity to local area networks (LANs) and / or larger networks, such as wide area networks (WANs). Such LAN and WAN networked environments are common in offices and companies, facilitating enterprise-scale computer networks such as intranets, either of which can connect to global communication networks, such as the Internet.

[0021] When used within a LAN network environment, a computer can be connected to the LAN through a wired and / or wireless network interface or adapter. The adapter can facilitate wired or wireless communication to the LAN, and this may also include a wireless access point (AP) placed on top of it to communicate with the adapter in wireless mode.

[0022] When used in a WAN networked environment, a computer can connect to a communication server on the WAN, including a modem, or via other means for establishing communication over the WAN, such as via the Internet. A modem, which may be internal or external and may be a wired or wireless device, can be connected to the system bus via an input device interface. In a networked environment, program modules described herein for the computer or a part thereof can be stored in remote memory / storage devices.

[0023] When used within a networked environment, either a LAN or a WAN, a computer can access cloud storage systems or other network-based storage systems in addition to, or instead of, external storage devices. Generally, the connection between the computer and the cloud storage system can be established via a LAN or WAN, for example, through an adapter or modem, respectively. Depending on the step of connecting the computer to the associated cloud storage system, the external storage interface can, with the help of the adapter and / or modem, manage the storage provided by the cloud storage system, as other types of external storage devices would. For example, the external storage interface can be configured to provide access to cloud storage sources as if they were physically connected to the computer.

[0024] As adopted herein, the term “processor” can substantially include, but is not limited to, any computing unit or device comprising a single-core processor, a single-core processor with software multithreading capability, a multi-core processor, a multi-core processor with software multithreading capability, a multi-core processor with hardware multithreading technology, a vector processor, a pipeline processor, a parallel platform, and a parallel platform with distributed shared memory. In addition, a processor can include an integrated circuit, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic controller (PLC), a coupled programmable logic device (CPLD), a state machine, discrete gate or transistor logic, a discrete hardware component, or any combination thereof, designed to perform the functions described herein. A processor can utilize nanoscale architectures such as molecular and quantum dot-based transistors, switches, and gates, but is not limited to, to optimize space use or improve the performance of user equipment. A processor may also be implemented as a combination of computing units. For example, processors may be tightly coupled, loosely coupled, or located remotely from one another, implemented together as one or more processors. Multiple processing chips or multiple devices may share the performance of one or more functions as described herein, and similarly, memory may be provided across multiple devices.

[0025] As an overview, various arrangements are described herein. For the convenience of explanation, methods (or algorithms) are depicted and described as a series of steps or actions. It should be understood and acknowledged that the various arrangements are not limited by the illustrated actions and / or the order of the actions. For example, actions may occur in various orders and / or in parallel, and with other actions not presented or described herein. Furthermore, not all illustrated actions may be required to implement the method. In addition, the method may, as an alternative, be represented as a series of interrelated states via a state diagram or events. In addition, the methods described hereafter may be stored on a manufactured product (e.g., a machine-readable storage medium) to facilitate the step of transporting and transferring such methodologies to a computer.

[0026] In one aspect, the system of this instruction collects data, for example, sensor data, for example In some aspects, the rule set processor processes a filter rule set only after at least one criterion is met. For example, a filter rule set may be processed after it has been verified that the sender of the filter rule set is an authorized entity. In some aspects, the system removes data that does not meet selection criteria. Selection criteria may include, but are not limited to, dynamically determined criteria, default criteria, and / or criteria established by the user. In some aspects, the system operates in multiple modes. In some aspects, the system automatically selects a mode based on data collected by a collection device, for example. In some aspects, the user selects a mode. In some aspects, a default mode is selected based on the desired use. In some configurations, the system includes a general mode, a target mode, and a context mode. Other modes may also be assumed and adapted by this teaching.

[0027] In one aspect, the general mode can be analogized to a surveillance camera in a store where sensor data can be collected when it is determined that some object of interest can occur within a specific time frame. In the general mode, the system collects all sensor data indicated by, for example, a rule set, a default set of sensors, or a user-supplied set of sensors. For example, since a particular application may require point cloud and image data to be collected in a certain geography and within a certain time frame, along with audio and chemical data, a user may choose to collect LIDAR data, camera data, audio data, and chemical data. Additionally or alternatively, the general mode may indicate, by default setting, that all available sensors are activated and all available sensor data are collected. In one aspect, in the general mode, the system preprocesses incoming data by filtering the data according to preselected criteria, such as, but not limited to, blurriness, signal-to-noise ratio, data quality, temporal filtering, or detection of defined objects. In some arrangements, further filtering is performed according to a preselected processing recipe for general data or according to specific rules that may be provided to the system. In the general mode, in one aspect, the data remaining after filtering is encrypted, optionally encoded, and then provided to the user for review. In some aspects, for example, when a person of interest is being sought, in the general mode, the user or system owner may be a private sector user, and any remaining data is made accessible to the system owner, and if such is desired, further object-level filtering and analysis are made available. In some aspects, in the general mode, the user may be a public sector user, and any remaining data is authorized and configured such that further object-level filtering and analysis are made available without making the data available for public use.

[0028] In some aspects, the general mode may be used to implement background subtraction. Such background subtraction is used in scenarios where a geographic area is specified with objects that are permitted or expected to exist, and the system detects exceptions within such areas and notifies the user that an exception has occurred. Such notification may include a step of providing image data and / or other data representing the exception. For example, the system may be configured to specify areas with restrictions or requirements, for example, providing a warning if a car is parked in a place where it should not be parked.

[0029] In one respect, the target mode enables the user to apply image recognition to the collected data. For example, if it is necessary to locate a specific person, a certain type of machine learning model used for this purpose is trained to sort through the collected data and find the person in the image. Those skilled in the art will understand that one of the many embodiments of a model suitable for face recognition is a neural network model called Deep Face. An exemplary face recognition process follows four steps: (1) face detection, where one or more faces in an image or video are located and marked with bounding boxes; (2) face matching, where the detected faces (and their location, size, and orientation) are normalized to match a database, for example, one relating to geometric shape and photometry; (3) feature extraction, where features are extracted from the matched faces that can be used for recognition; and (4) feature matching of feature vectors representing faces against one or more known faces in a prepared database (e.g., a database of registered users). In one aspect, the system has separate modules or programs for each of the four steps, or combines some or all of the steps into a single process. Similarly, if it is necessary to locate objects such as license plates, a certain type of machine learning model used for this purpose is trained to sort through collected data and find specific license plates. Those skilled in the art will understand that one of the many embodiments of a model suitable for license plate detection and identification is sold by Plate Recognizer. In an exemplary license plate detection and identification process, a first object detection model is used to recognize images of cars or other vehicles in multiple images and / or videos. In one aspect, a detection model is used to identify license plates in images of cars or other vehicles. The detection model does not have to be a machine learning model. If a machine learning model is used, for example, a neural network such as YOLO or SSD (both defined below) may be trained to detect license plates.Optical Character Recognition (OCR) is performed on the detected license plate and can convert images into text. In one aspect, in the subject mode, when data collection and data processing are performed in processors that are geographically remote (relative to each other), at the data collection site, the data undergoes a first-pass filtering process. The first-pass filtering process sorts through the data, for example, identifies bounding boxes for objects, sorts the objects from the remaining portion of the data, and the latter are omitted. The data within the bounding boxes is transmitted from the data collection module to a processing module that subjects the data to further filtering. In one aspect, the processing module selects a machine learning model that would be appropriate for the selected object type such that when the collected data arrives at the processing module, the object-specific machine learning model is applied to the data. Potential matches are located, and non-matches are deleted. The potential matches are provided, for example, to a user interface of a handheld device, a tablet, or a laptop, to a log file, and / or to a user interface of a local or remote system operator desktop. In various configurations, multiple filtering passes are executed within the same processor. Additionally or alternatively, multiple filtering passes are performed at the data collection site. Additionally or alternatively, multiple filtering passes are performed remotely from the data collection site. Additionally or alternatively, multiple filtering passes are performed on various processors local to and remote from the data collection site.

[0030] Various machine learning models are known to those skilled in the art and, in non-exclusive and non-exclusive embodiments, may include clustering, dimensionality reduction, ensemble methods, neural networks (e.g., convolutional neural network models), and deep learning, transfer learning, reinforcement learning, natural language processing, and word embeddings. Many suitable techniques for object detection and recognition are readily understood by those skilled in the art and, in non-exclusive embodiments, include region-based convolutional neural networks (R-CNN), fast R-CNN, faster R-CNN, region-based fully convolutional networks (R-FCN), oriented gradient histograms (HOG), single-shot detectors (SSD), spatial pyramid pooling (SPP-net), and You Only Look Once (YOLO).

[0031] In one embodiment, in the target mode, the system according to this disclosure receives electronic authorizations from an authorized entity, which may be used to create and / or extend filter rule sets. The new filter rule sets can, in effect, authorize an autonomous vehicle to keep watch for vehicles of interest, e.g., a Black Hummer H1 with a specific license plate, by passing the filter rule set to a collection device. The collection device may be embedded within the autonomous vehicle of this teaching. In addition, or alternatively, the collection device may be located remotely, or at least partially remotely, from the autonomous vehicle and coupled to it in a communicative manner. As the autonomous vehicle collects data, the data is compared against a filter rule set, e.g., a list of vehicles included in all active authorizations. In some aspects, the data associated with matching vehicles enumerated in the active authorizations is stored, passed to a processing module, and, potentially, made available to law enforcement agencies, based on further processing performed by the processing module. When a vehicle of interest is detected, the law enforcement agencies listed on the authorizations are notified that the vehicle has been detected and are provided with the date / time of such detection. In one respect, law enforcement agencies can securely view and, in some cases, download images of vehicles of interest.

[0032] In some configurations, the system according to this disclosure may operate in scene mode. Scene mode enables the user to apply focus scene rules to the collected data. In aspects similar to those relating to operation in object mode, a specific focus scene (similar to a particular object of interest) may appear at any given time from the perspective of the data collector. The criteria for recognizing a focus scene do not have to be specific to a particular location, but instead may provide general characteristics of what such a scene could be. For example, if a machine learning model is trained using general characteristics of arson scenes, the data collector will recognize possible arson scenes. In scene mode, the trained machine learning model is used either within the data collector or in an array where two data processing paths are available both within the data collector and the processing system, for example, one in the data collector and the other in the processing system. Scenes identified by the machine learning model are provided to the user, computer, log files, and / or various types of displays, and the remaining data is deleted.

[0033] In some aspects, the system of this instruction includes a collection processor configured to receive sensor data and encrypt the received sensor data. In various aspects, the collection processor and one or more sensors are mounted, for example, on an autonomous vehicle, a utility pole, and / or a drone, and / or the collection processor and sensors are transported by a person or animal. In some aspects, the collection processor is configured to execute coded instructions stored in its memory or in memory coupled to the collection processor in order to filter the sensor data. In some aspects, data to be filtered and removed is deleted. In some aspects, sensor data that the filter indicates has a high probability of matching, for example, a probability of matching at or above a given threshold probability, is reserved. To perform filtering, in some configurations, the collection processor locates features in the received sensor data and, where possible, matches those features to any of the items in a list of features that are considered likely to be of interest, provided according to a set of filter rules, for example, in a trigger list. In some aspects, the trigger list is created from rules established, for example, by law enforcement and / or local government officials, for example, but not limited to. In one aspect, the trigger list includes a subset of the entire set of rules so that the processing requirements on the acquisition processor can be reduced or minimized in order to reduce power consumption. In one embodiment, the trigger list includes facial data of a suspected individual or missing person, and the acquisition processor looks for a match between the facial data in the trigger list and the received sensor data and / or filtered sensor data. Those skilled in the art will readily understand that many types of sensor data can be collected, and that the trigger list can include triggers for many types of data. For example, if the trigger is an object, object detection can be performed using techniques by various methods known to those skilled in the art, including, but not limited to, R-CNN, Fast R-CNN, Faster R-CNN, R-FCN, HOG, SSD, SPP-net, and YOLO.Other types of sensor signals and / or other types of data may be detected and collected, in non-limiting embodiments, including LIDAR signals, radar signals, ultrasound, optical camera data, audio data (e.g., voice or music data), chemical data, infrared signals, magnetic or near-field waveforms, electromagnetic or radio frequency waveforms, point cloud data, bitmaps, alphanumeric data (e.g., car license plate numbers), video data, detected faces, and other types of detected objects.

[0034] In one aspect, the collection processor is configured to ensure that the collected data is maintained in a secure state, whether the data is stored locally or transmitted to a remote location, such as a processing station. For example, the collection processor is configured to encrypt all data as it is received. In another aspect, the collection processor is configured to re-encrypt decrypted data, i.e., elements of the data selected for transmission to the processing station, within the transmission chain. As will be understood by those skilled in the art, various known cryptographic techniques, such as Advanced Encryption System (AES) 256-bit encryption, may be used in non-limiting embodiments. In addition, or alternatively, the collection module authenticates the data by ensuring that the data is digitally signed before it is transmitted, for example, to enable a receiver to prove the identity of the sender upon receipt of the data. In addition, or alternatively, the collection module associates metadata, such as time and / or location data, with the transmitted data. In addition, or alternatively, the data to be transmitted to the processing station is compressed to conserve bandwidth. Compression may be performed, as those skilled in the art will understand, in non-limiting embodiments, using any of various known standardized methods such as vocoding for audio data, ITU-T H.264, HEVC, or VVC for video data, Huffman coding, lossless compression, or lossy compression. In addition, or alternatively, a checksum is applied to the data to be transmitted to the processing station in order to enable the receiver at the processing station to detect any errors caused by the transmission medium. In addition, or alternatively, the data to be transmitted to the processing station is encoded so that the receiver can decode the data to correct for errors caused by the transmission medium.As those skilled in the art will understand, error correction encoding may be performed using various known techniques, including, in non-limiting embodiments, block coding, e.g., Reed-Solomon coding, convolutional encoding, turbo encoding, low-density parity-check (LDPC) encoding, or polar encoding. Those skilled in the art will also understand that the error correction encoding format adopted may depend on the communication protocol used for transmission. In some aspects, when compression, encryption, and error correction encoding are performed on the data prior to its transmission, the data may first be compressed, then the compressed data may be encrypted, and then the encrypted compressed data may be error corrected.

[0035] In some sequences, the acquisition processor uses one or more of a variety of known machine learning models to perform feature extraction and matching, with computations performed on the autonomous vehicle (e.g., at an edge node rather than in the cloud). In some aspects, the feature-matched target set of such features is predefined within a rule set. The acquisition module applies tracking during object detection, including, for example, temporal filtering of objects. Such filtering includes a first type of filtering and a second type of filtering that may be performed by a remote processor at a processing station (e.g., in the cloud). In some aspects, the first type of filtering includes coarse-grained filtering for the second type of filtering. In some aspects, the acquisition module utilizes a two-dimensional (2D) bounding box tracking mechanism, such as Generalized Union Crossing (GIOU) tracking, to track objects and their bounding boxes from each acquired image frame to subsequent acquired image frames. In other sequences, the acquisition module tracks objects using the Joint Probabilistic Data Association Filter (JDAF) algorithm, linear velocity prediction techniques, and / or a combination of the two. In some aspects, the acquisition module crops one or more of the images within the bounding box of the object of interest, or all of them, as defined by the rule set. The acquisition module transmits the data to a remotely located processing station or a co-located processing station, at least one such cropped image related to the detected and tracked object of interest. In some aspects, the acquisition module compresses, encrypts, and / or encodes the cropped images before transmitting them.

[0036] A system according to several aspects of this disclosure includes a processing module (e.g., a processing station, or in a processing station) configured to receive and process filters or triggers used to sort collected data. In some aspects, the processing module receives inputs such as, for example, triggers and associated permissions to search for triggers. Triggers include, for example, objects, images, odors, sounds, scenes, and / or other sensory data. In addition, or alternatively, the processing module receives such identification information as, for example, the location of one or more scenes of interest, the duration of one or more sounds of interest, and / or one or more notifications such as, for example, communications from a collection device or a human, or automated messages, including notifications of matching or high-probability matching between collected image data and scenes or objects of interest. In some embodiments, the processing module is configured to receive and execute instructions provided via inputs received from an end-user application. In some aspects, the end-user application is used to request a set of rules to be applied by the processing module. In various embodiments, the processing module itself includes a manager program. In some embodiments, the manager program is a cloud-based manager. The manager program is designed to maintain, among other things, data privacy, data security, storage chain control, and / or audit trails (e.g., time or data stamps, etc.). In some aspects, the manager program is rule-set-agnostic; that is, the manager program is not limited to processing rule sets encoded in any one specific format.In some configurations, the manager program includes instructions that enable the provision of application programming interfaces (APIs) for privacy, storage chains, and / or audit rules to be configured by users such as police, government authorities, the National Security Agency, or corporate customers with special privacy requirements, in non-exclusive embodiments.

[0037] In various configurations, processing modules and collection modules (e.g., collection processors) communicate with each other, for example, through a gateway. In one aspect, the gateway establishes a web service between the collection modules and the gateway, as will be understood by those skilled in the art, and uses the web service to establish communication between the collection modules and the gateway. In some configurations, the gateway is the web server for the web service, and the collection modules are the web clients for the web service. In another aspect, data is communicated between the collection modules and the gateway using the web service.

[0038] Processing modules, in various configurations, are configured to decrypt data received from collection modules using encryption keys associated with the encrypted data received from the collection modules. In addition, processing modules are optionally configured to re-encrypt data received from the decrypted collection modules. In some configurations, processing modules verify and track the storage chain and / or digital signature associated with the incoming data and / or enable the digital signature of some or all outgoing data transmitted by the processing module. In various configurations, processing modules receive a rule set and, at least partially, prepare, based on it, one or more subsets of the rule set (e.g., trigger lists) for use by the collection processor and transmit them to the collection modules. In various configurations, processing modules apply fine-grained filters to the broadly filtered data received from the collection modules and apply rule sets to the filtered data. The filtered results, including at least matched data and matching notifications, are transmitted to, for example, authorized individuals, log files, cloud-based systems, laptops, handheld devices, desktops, and / or tablets.

[0039] The processing module, in various aspects, uses one of various machine learning models to subject the data received from the collection module to a finer-grained feature matching than the relatively coarse feature matching calculation performed by the collection module, for example, a more detailed version of feature matching. For example, the processing module determines data that satisfies trigger requirements described in a rule set for the filter. In some aspects, the processing module includes at least an extraction algorithm and a recognition algorithm. In some aspects, the extraction algorithm is used to decrypt and / or decode sensor data that has been pre-collected and filtered by the collection module. In some aspects, the extraction algorithm includes at least a neural network algorithm configured to receive the decoded and filtered sensor data and generate bounding boxes containing the desired sensor data. In some aspects, the extraction algorithm processes the detected sensor data provided by the neural network algorithm. In some aspects, the recognition algorithm subjects the detected sensor data to further processing, such as, but not limited to, an optical property recognition algorithm. In addition, or alternatively, the recognition algorithm filters the data resulting from text matching, for example, between text generated from the data and trigger values ​​such as license plate numbers described in a set of filter rules, classifying the text as matching / not matching. In some respects, the processing module stores the results from further processing. In some respects, the storage is secure. In some respects, the processing module encrypts the storage area so that the data is accessible only to authorized users based on the received trigger data. In some respects, the processing module sends a notification by sending text to an authorized user when a match is found between the data and the trigger database, in a non-restrictive embodiment. In some respects, the processing module allows authorized users to view and download the matched data and the location where the matched data is collected.In some respects, browsing is secure. In other respects, data that does not provide matching according to the trigger list will not be made available to authorized individuals or other third parties in order to protect the privacy of those who are not of interest. Such data may be deleted, or such data may be encrypted and stored. Those skilled in the art will understand that the systems provided by this disclosure do not need to be limited to detecting and / or recognizing only facial features. Those skilled in the art will understand that machine learning models can be trained to recognize many types of objects, including, but not limited to, license plates, automobiles, animals, and consumer goods.

[0040] The systems described in this disclosure may be used in a variety of different ways, each raising privacy and security concerns addressed by the system's architecture. For example, when the system is used to locate individuals for whom a search authorization has not yet been processed, the system may be operated by an autonomous vehicle, and for example, the system may collect a substantial number of images of non-suspect individuals. The exemplary system prevents the location and / or activities of non-suspect individuals from being made available to law enforcement. Filtering and encryption by the collection and processing modules may enable the provision of images to law enforcement that substantially resemble the appearance of the suspect. On the other hand, the systems described in this disclosure may be used in general to scan the environment without searching for any specific individual or item of interest. In the act of scanning the environment, the system may detect evidence that a crime has been committed or a crime that is being committed. The exemplary system ensures that the captured images are not unsuitable as evidence, and / or that the captured images provided to authorities do not contain data that could involve innocent people. The system described in this disclosure performs the above by, for example, evaluating incoming data based on rules set by the authorities, but not limited to these rules.

[0041] As will be readily apparent to those skilled in the art, AES-256 encryption may be used within the collection and / or processing modules of this disclosure when symmetric key encryption is required, such as for encryption of stored data for data stores such as object, relational, directory, and / or search data stores. The AES-256 encryption algorithm may also be used to encrypt message blocks exchanged between the collection and processing modules over a network. Such messages may also, or alternatively, be encrypted using TLS 1.2 encryption used for computer network channels.

[0042] Furthermore, as will be understood by those skilled in the art, messages containing data exchanged between a collection module and a processing module as provided in this disclosure may be digitally signed, and a cryptographic hash may be generated for each such message. The recipient of the message may decrypt the message using the sender's public key certificate. The cryptographic hash may be regenerated at the recipient. Both cryptographic hashes may be compared to verify the authenticity of the message. If the two cryptographic hashes match, the message may be determined to be valid. RS-2048 encryption may be used for digital signing.

[0043] In some aspects, access to data requires user authentication and authorization. For example, multi-factor authentication using at least two strong authentication codes may be desirable in situations involving law enforcement. Authenticated users may be authorized to view specific data, such as defined, using role-based access control that limits access based on, for example, subject, time frame, geography, and / or various other parameters. [Brief explanation of the drawing]

[0044] Non-exclusive and non-exclusive aspects of this disclosure are illustrated with reference to the following figures, and similar reference numbers refer to the same parts throughout the various figures unless otherwise specified.

[0045] [Figure 1] Figure 1 is a flowchart illustrating the flow and steps of various aspects of this disclosure.

[0046] [Figure 2] Figure 2 is a flowchart illustrating the flow and actions of various aspects of this disclosure.

[0047] [Figure 3] Figure 3 is a schematic block diagram of the system from various aspects of this disclosure.

[0048] [Figure 4] Figure 4 is a message flow diagram illustrating the message flow from various aspects of this disclosure.

[0049] [Figure 5] Figure 5 is a schematic block diagram of the exemplary system described in this instruction.

[0050] [Figure 6] Figure 6 is a flowchart illustrating the flow and steps of various aspects of this disclosure.

[0051] [Figure 7] Figure 7 is a flowchart illustrating the flow and steps of various aspects of this disclosure. [Modes for carrying out the invention]

[0052] Detailed explanation In some configurations, the system may perform first pass processing, assuming that first and second pass processors will transfer data between them in the manner depicted in Figure 1. In some aspects, second pass processing may be performed according to the manner depicted in Figure 2. Referring to Figure 1, if it is not time to transmit data in action 1851, for example, if the desired amount of data has not been collected, or the time limit for data collection has not expired, or some other known criterion for discontinuing data collection has not been met, then the flow control proceeds to action 1852. In action 1852, if there is no more data to be processed, then the first pass processing terminates. If there is more data to be processed in action 1852, then the control flow proceeds to action 1857. In action 1857, the system receives a desired mode (for example, from the second pass processor). The mode may be a default mode established by the system user, determined by the application selection made by the user, determined by one or more sensors in a set, or selected in any of various other suitable ways. The mode may be set by the system user, but alternatively, the mode may be dynamically determined by the data acquisition system. The control flow then proceeds to action 1859.

[0053] In action 1859, the system activates one or more sensors based on the determined mode, and receives and encrypts data from one or more sensors. If the mode is predetermined and sensors are activated, the system continues to receive and encrypt data. The control flow then proceeds to action 1861. In action 1861, if the mode is the general mode, the control flow proceeds to action 1863. In action 1863, the system accumulates the sensor data received from the activated sensors. In some configurations, the system supports dynamic adjustment of the set of activated sensors based on, for example, the occurrence of a sensor failure or whether at least some of the collected sensor data indicates that other sensors should be activated. If, in action 1861, it is determined that the mode is not the general mode, the control flow proceeds to action 1865.

[0054] In action 1865, if the mode is the target mode, the control flow proceeds to action 1867. In action 1867, the system determines the type of the desired target. In a non-limiting embodiment, the desired target may be a human, an animal, or an object. The control flow then proceeds to action 1869, where the system may, in some configurations, select a trained machine learning model, the selection of which may be at least partially based on the type of target. The control flow then proceeds to action 1871, where the system applies the selected and trained machine learning model to the sensor data. In some aspects, in the first pass of processing, the selected machine learning model is trained to identify sensor data that generally satisfies the characteristics of the target, but more specifically, fails to satisfy the characteristics of the target (e.g., more detailed, finer, or finer-grained characteristics). In some aspects, the first pass processing and the second pass processing are combined to enable the system to identify a specific target in a single pass. Furthermore, as those skilled in the art will understand, it is possible to adjust the first pass processing relative to the second pass processing (and vice versa) to achieve optimal results based on the type of subject. In other words, the relative coarseness of the first pass filtering and the relative fineness of the second pass filtering may be adjusted or coordinated relative to each other as desired. An embodiment of the first pass processing when the subject is a human is to identify all sensor data that meet the criteria of human-ness according to a trained machine learning model and discard the rest of the data. If it is determined in action 1865 that the mode is not the target mode, the control flow proceeds to action 1873.

[0055] In action 1873, if the mode is a scene mode, the control flow proceeds to action 1875. In action 1875, the system may determine the type of scene to be desired. In some sequences, possible scenes of interest, and in non-limiting embodiments, general characteristics of a crime scene, may be known in advance. In other aspects, possible scenes of interest, and in non-limiting embodiments, general characteristics of a crime scene, may be supplied by the system user. The control flow then proceeds to action 1877, where the system selects one or more machine learning models based at least in part on the type of scene of interest. The control flow then proceeds to action 1871, where the system applies one or more selected machine learning models, trained to identify specific types of scenes, to the collected sensor data. In addition, or alternatively, the step of applying multiple machine learning models to the collected data is performed as part of a second path filtering.

[0056] In action 1851, if the desired amount of data has been collected, or the data collection time limit has expired, or some other criterion for discontinuing data collection has been met, the flow control proceeds to action 1853. In action 1853, the system encodes the generated data by filtering, for example, by filtering using one or more machine learning models in some configurations. The control flow then proceeds to action 1855, where the system transmits the data to a second path processor. The control flow then returns to action 1851, where the timer for discontinuing data collection may be reset, and data collection and processing continue, if applicable.

[0057] In some aspects, where a second path processing is required, the system performs the second path processing in the manner depicted in Figure 2. Those skilled in the art will understand that both the first and second path processing may be performed by a single processor executing coded instructions. In some configurations, the second path processor interfaces with the system user, for example, through an application, to allow the user to interact with the second path processor. The user interface may be arbitrary and, at least by default, may be performed by recipes and / or dynamically determined criteria.

[0058] Referring to Figure 2, in some configurations, in action 1951, the system determines the data acquisition interval. The data acquisition interval may be a default value, or may be defined by the system user, or, in a non-limiting embodiment, may be dynamically determined based on the number of available sensors or the number of available sensor types, or the number of each type of available sensors. The control flow then proceeds to action 1953, where the system determines the desired mode. In various aspects, the system user may set the desired mode, or the desired mode may be dynamically determined by the system, or the desired mode may be established, at least in part, based on information requested by the system user. The control flow then proceeds to action 1955, where the system receives rules governing which parts of the collected data are relatively more significant. For example, the rules may include designation of an image of a person of interest or a scene of interest. The rules may be established, for example, by law enforcement or other interested authorities or individuals. The control flow then proceeds to action 1957.

[0059] In Action 1957, the system selects one or more machine learning models based on at least a determined mode. In addition, or alternatively, one or more machine learning models are selected at least in part based on the received rules or other criteria that a person skilled in the art would understand. In some configurations, models other than machine learning models are deployed. The control flow then proceeds to Action 1959. In Action 1959, the system trains one or more selected machine learning models based on the received rules. For example, if specific people are the focus, the selected machine learning model may be trained to find matches between collected data and images of the focus. In configurations where multiple processors are deployed, the control flow proceeds to Action 1961, where the system provides a desired mode to a first path processor, which is coupled to or associated with a data acquisition module, for example. The control flow then proceeds to Action 1963, where the system instructs the first path processor to begin data acquisition. The control flow then proceeds to action 1965, where the system determines whether the data acquisition time interval has expired. If the data acquisition time interval has expired, the control flow proceeds to action 1967. In action 1967, the second path processor receives data from the first path processor, and, if applicable, the second path processor either decodes the received data and / or decrypts the received data or encrypts the received data and / or encodes the received data for transmission or provision. The control flow then proceeds to action 1969.

[0060] In Action 1969, the system determines whether the operating mode is a general mode and not, for example, a target mode or a scene mode. If it is determined that the operating mode is a general mode, the control flow proceeds to Action 1971. In Action 1971, the system provides the data to the system user for evaluation, or, at least partially, depending on the application, the system subjects the data to further processing or filtering. If it is determined in Action 1969 that the operating mode is not a general mode, i.e., the operating mode is determined to be one of the target mode or a scene mode, the control flow proceeds to Action 1973. In Action 1973, the system applies a trained machine learning model to the data and generates matched data by determining whether any of the following matches exist in the data as a subset of the data: for a specific desired target, for a scene of interest to the system user, or for a scene determined by the received rules (in a non-limiting embodiment, a scene indicating that a crime has been or is being committed). The control flow then proceeds to Action 1975, where the system removes any or all data that is not matched data. The control flow then proceeds to action 1971. In some aspects, additional data is collected after the data has been evaluated by the system user. In some aspects, the operating mode and / or rules are modified before data collection resumes.

[0061] Referring to Figure 3, the system 100 in various aspects can selectively process images based on predetermined criteria. The system 100 includes at least an acquisition module 131 and a processing module 147. In some aspects, the acquisition module 131 is coupled to the processing module 147 by a communication medium 119. The communication medium 119 may be a wired connection such as Ethernet®, or it may be a wireless connection such as WiFi or a cellular or wide-area network. In some arrangements, the data may be encrypted during transmission across the communication medium 119. In some configurations, the acquisition module 131 and / or the processing module 147 are implemented as processors (not shown) that execute coded instructions stored in memory (not shown) that is accessible by them and / or integrated with them. In some aspects, the acquisition module 131 and the processing module 147 are located far apart from each other. In other aspects, the acquisition module 131 and the processing module 147 are juxtaposed. In some respects, the collection module 131 and the processing module 147 are implemented as a single processor.

[0062] In one configuration, the acquisition module 131 includes a feature detector 105, a feature matching processor 107 coupled to or integrated with the feature detector 105, a data filter 109 coupled to or integrated with the feature matching processor 107, a data compressor 111 coupled to or integrated with the data filter 109, a data storage encryption module 113 coupled to or integrated with the data compressor 111, and a transmission chain 115 coupled to or integrated with the data storage encryption module 113. The data filter 109 performs a first type of filtering on the received data. In some configurations, the data storage encryption module 113 encrypts and digitally signs the data so that the receiver can verify the identity of the sender and / or determine whether the received data has been altered. The transmission chain module 115 provides forward error correction coding and / or modulation of the filtered, encrypted, and digitally signed sensor data for transmission over the communication medium 119. In one aspect, the transmission chain module 115 provides forward error correction coding and / or modulation of metadata (e.g., timestamps and / or GPS location) for transmission over the communication medium 119. In some embodiments, the data receiver uses the metadata (e.g., GPS location and timestamps) to verify whether the collection module 131 was present at the indicated location at the indicated time. Any or all of the feature detector 105, feature matching processor 107, data filter 109, data compressor 111, storage data encryption module 113, and / or transmission chain 115 may be implemented as one or more processors, microcontrollers, or state machines that execute code stored in hardware (e.g., ASIC or FPGA), software or firmware modules, or in memory.The data collection module 131, and the feature detector 105, feature matching processor 107, data filter 109, data compressor 111, storage data encryption module 113, and transmission chain 115 can function substantially for the various features shown in Figures 1 and 2, as described above.

[0063] In some respects, the processing module 147 includes one or more of the following: a data decryption module 135, a storage processor 137 coupled to or integrated with the data decryption module 135, a data filter 139 coupled to or integrated with the storage processor 137, a rule set processor 141 coupled to or integrated with the data filter 139, a storage data encryption module 143 coupled to or integrated with the rule set processor 141, and a signature processor 145 coupled to or integrated with the storage data encryption module 143. The data decryption module 135 performs decryption using public and private keys. Those skilled in the art will understand that the private key is a key that resides at all times, along with an authorized entity that uses the key to decrypt received data. The storage processor 137 tracks the storage chain. The data filter 139 performs a first type of filtering on the received data. The signature processor 145 may verify the digital signature associated with a data packet prior to any step that uses the data, in order to prove that the data originates from a trusted source. Any or all of the data decryption module 135, storage processor 137, data filter 139, rule set processor 141, data storage encryption module 143, and / or signature processor 145 may be implemented as one or more processors, microcontrollers, or state machines that execute code stored in hardware (e.g., ASIC or FPGA), software or firmware modules, or in memory. The processing module 147, and feature data decryption module 135, storage processor 137, data filter 139, rule set processor 141, data storage encryption module 143, and signature processor 145 function substantially as described above for various aspects of Figures 1 and 2.

[0064] In one aspect, the data collection module 131 is configured to receive data from one or more sensors 103. The types of sensors 103 that may be available to the data collection module 131 may depend on the environment of the system 100. For example, the sensors 103 may be mounted on a device such as a remotely controlled robot, or abbreviated as a bot, autonomous bot, or autonomous vehicle (AV) 102, configured to run the system 100, and / or the sensors 103 may include optical cameras, laser devices, ultrasonic sensors, weather sensors, LiDAR sensors, radar sensors, infrared sensors, and / or near-field sensors, etc. An exemplary AV is described, for example, in the system shown and described in U.S. Patent Application No. 16 / 926,522 (Patent Attorney No. AA291), filed July 10, 2020, titled "System and Method for Real Time Control of an Autonomous Device". In some configurations, the device on which the system 100 is mounted is mobile. Data received from one or more sensors 103 is provided to the feature detector 105. The collection module 131 receives a trigger list from the processing module 147 via the communication medium 119.

[0065] In one configuration, a rule set provider source 123 is coupled to a processing module 147. The rule set provider source 123 provides the processing module 147 with one or more rule sets. The rule set provider source 123 could be, for example, a data store under the control of a law enforcement agency or other local government that tracks subjects of interest. Embodiments of subjects of interest may include, but are not limited to, people, vehicles, and / or tangible devices. In one aspect, the rule set includes information about subjects of interest that can be used by the system 100 to locate them. In one embodiment, the rule set is provided to a rule set processor 141, from which a subset of rules, e.g., a list of triggers, is selected, at least in part, based on, for example, the location, time, and / or any other factors that may make a selected subset of rules relatively more useful or applicable, e.g., the location, time, and / or selected subset of rules of the system 100. In one aspect, the rule set processor 141 is coupled to a communication medium 119 via a transmitter (not shown) so that one or more trigger lists are provided to the collection module 131, specifically to the feature detector 105 via a receiver (not shown). Those skilled in the art will understand that one or more trigger lists may be provided directly to the collection module 131, in addition or alternatively. In one aspect, the processing module 147 is configured to provide at least one of the matching notification and the matched sensor data to the application 133 for use by an authorized actor. In one sequence, the signature processor 145 is configured to provide at least one of the matching notification and the matched sensor data to the application 133 via a transmitter (not shown) for use by an authorized actor.

[0066] In some aspects, data is stored in legally permissible locations; for example, U.S. data is not stored on servers located outside the U.S. In various aspects, data collected by one or more sensors 103 resides in RAM memory (not shown) until it must be provided to feature detectors 105. In some aspects, each data source reserves its own private key, which is used to encrypt data transmitted from that source. In some aspects, an authorized system user or entity uses the private key in combination with the public key to decrypt data received from an authorized data source. In various configurations, the storage chain may include one or more of the following exemplary actors: collection devices at rest, data transfer mechanisms, cloud-based receiver services, cloud-based detection filter services, cloud-based notification services, and end users (e.g., law enforcement agencies).

[0067] Figure 4 illustrates the message flow 200 in illustrative aspects. It will be understood by those skilled in the art that any or all of the various entities shown in Figure 4 may be either physical or logical entities, may be juxtaposed with each other or located at a distance from each other, and / or may be implemented as a single entity or processor. It will also be readily understood by those skilled in the art that any or all of the various messages shown in Figure 4 (in the context of different entities rather than a single entity) may be transmitted / received over any known communication medium, including, but not limited to, wired (e.g., Ethernet®) and wireless (e.g., WiFi, cellular, satellite) communication media. Referring to Figure 4, the collection device 201 transmits an image message 202 to the target type filter module 203. The target type filter module 203 attempts to detect a target type (determined as described below), e.g., a face or a license plate, within the received image message 202. If a target type is detected, the target type filter module 203 transmits a type-matched image message 204 to the target filter module 205. The target filter module 205 attempts to detect a specific target (determined as described below) within the received type-matched image message 204, such as the face of a specific person or a license plate with a specific license plate number. If a specific target is detected, the target filter module 205 transmits the target-matched image message 206 to an authorized agent 207 (e.g., a law enforcement agency or other local government).

[0068] Continuing to refer to Figure 4, the authorized agent 207 transmits the rule set message 208 to the rule set manager module 209. The rule set manager module 209 attempts to verify (e.g., authenticate) the rule set in the received message 208 (which contains rules that define, for example, a target or item, a set of criteria used to detect the target or item, etc.). If the rule set manager module 209 verifies the rule set in the rule set message 208, it transmits the verified rule set message 210 to the persistent storage module 211. The rule set manager module 209 also creates a target type filter based on the verified rule set, at least partially. The rule set manager module 209 transmits the type filter message 212 to the persistent storage module 211. The rule set manager module 209 also creates a target filter based on the verified rule set, at least partially. The rule set manager module 209 transmits the target filter message 214 to the persistent storage module 211. The persistent storage module 211 transmits the received type filter message 212 to the target type filter module 203, which uses the received type filter message 212 to detect the target type in the received image message 202. The persistent storage module 211 also transmits the received target filter message 214 to the target filter module 205, which uses the received target filter message 214 to detect a specific target in the received type-matched image message 204.

[0069] Referring here to Figure 5, data arrives from the sensor 301 to cause the initial processor 303 to perform initial processing. In one aspect, the sensor 301 includes mobile and / or fixed sensors. In one aspect, mobile sensors are mounted on vehicles such as, for example, wheeled vehicles and / or autonomous vehicles, and / or drones, or on humans / animals. In one aspect, fixed sensors are mounted in stationary locations, such as, for example, roadways, traffic lights, signal lights, traffic signs, buildings, and / or monuments. The sensor 301 may include, for example, visual sensors such as cameras, signal sensors such as lidars and ultrasonic sensors, audio sensors, tactile sensors, and others. With respect to the initial processing 303 to be performed, the rule set 305 is made available from the trigger list 307, which is used, for example, to match faces and license plates. Data processing, in several aspects, includes compression, encryption, and encoding, followed by feature detection, feature matching (using rule sets), and a first type of filtering, such as, but not limited to, coarse-grained filtering. The data is encrypted at rest and digitally signed before transmission, enabling the receiver to verify the sender's identity upon reception. The data includes metadata, such as timestamps and GPS location. The processed data is transferred and / or stored for use at any location. For data transfer, only human-readable data is transmitted upon matching; otherwise, encrypted, signed, and encoded (raw) data is transmitted during transit. The processed data is received by a cloud processor 311, for example, the data is decrypted using a secret key and tracked using a storage chain strategy, digital signature, and metadata. The data is further filtered, and, for example, fine-grained filtering is applied. The cloud processor 311 receives a rule set 313 from a local government, processes the rule set 313, and provides an updated rule set 319 to a trigger list 307. The cloud processor 311 provides matching notifications and matched data 317 to the authorized agent 315.In the cloud processor 311, data is encrypted at rest, and digital signatures are verified before the data is used to prove that the data originates from a trusted source. The cloud processor 311 and / or initial processor 303 store the data. During data storage, data is stored at rest within a legal location; for example, US data cannot be stored on servers outside the US. Data used by the sensor 301 remains in RAM until they need to be transmitted, at which point they are encrypted. Each authorized data source has its own private key, which is used to encrypt the date from that source. Each authorized user / entity has a unique public key, which enables the user / entity to decrypt data received from authorized data sources. Exemplary actors along the data storage chain may include, but are not limited to, bots at rest, data transfer mechanisms 309, the cloud processor 311 (receivers, detection filters, and notification services), and end users, such as law enforcement agencies. For data integrity, each data source digitally signs each data packet with a unique key. Each data recipient verifies the digital signature for each data packet to confirm that the packet originates from an authenticated and authorized sender and that the packet was not tampered with during transit. Each data recipient verifies the data using contextual data; for example, if a matched image is from GPS coordinates x, y, the data recipient verifies that the patrolling bot was present at that location at the time reported in the matching result. Private keys, which are required for them to decrypt the data and which authorized parties are authorized to verify, exist with the authorized parties.

[0070] Referring here to Figures 6 and 7, a method for determining desired information from the configuration of this instruction is shown. The method in Figure 6 is described in terms of a collection device that collects sensor data and a processing device associated with a user or another processor that provides the retrieval information. In Figure 6, method 600 for determining desired information when receiving rules from an authorized agent includes, but is not limited to, the step of securely receiving (602) at least one rule from an authorized agent. In one aspect, the authorized agent is a law enforcement officer, and the rule is, for example, an authorization document involved in locating a vehicle, and the authorization document includes a description of the desired vehicle by make, model, and license plate number. Method 600 includes the step (604) of updating at least one rule database with at least one rule. In one aspect, the rule database includes, for example, information for detecting vehicles in general, specific make / model / type of a vehicle, general license plate information, and specific license plate information. The rule provided by the law enforcement officer provides specific information about a particular vehicle. When rules are added to the rule database, the database is expanded so that searches related to specific vehicles are possible. Method 600 includes the step (606) of securely transmitting the rule database to a collection device. Processing devices and collection devices may be placed side by side, but security measures such as encryption at rest can ensure that privacy concerns regarding rules and search data are not compromised. When processing devices and collection devices are communicating over a network, for example, the step of securely transmitting a message including legal authorization documents may include features such as encryption and man-in-the-middle sabotage. Method 600 includes the step (608) of securely receiving sensor data from a collection device. The collection device may encrypt in-place and encrypt its transmission of collected data, for example, images of vehicles. In one aspect, the collection device uses the rule database, in particular rules provided by law enforcement officers, to perform a "rough" filtering of the collected data.This step, in particular, reduces the amount of data encrypted and transmitted by the collecting device to the processing device. A coarse filter can, for example, exclude data that is not a vehicle. In one aspect, the data exclusion step is defined as deleting all data that does not meet the filter criteria. This step is optional, but protects the privacy of vehicle owners not associated with licenses and protects law enforcement from privacy violation complaints. The coarse filter can be adjusted to exclude data that is not a vehicle of the desired make / model / type and / or vehicles that do not have license plates. The filter can be adjusted according to the processing capacity of the collecting device and, where applicable, the transmission rate of the communication link between the collecting device and the processing device. Method 600 includes a step (610) of securely storing the received coarsely filtered data. In one aspect, storage is not required. However, if the data is stored, storage encryption protects the data from unauthorized access and thus protects the privacy of vehicle owners, for example. Method 600 includes the step (612) of securely storing and applying a fine-grained filter to broadly filtered data. For example, if the data includes vehicles of a desired make / model / type, the fine-grained filter can further examine the data in relation to matching for a desired license plate. Method 600 also includes the step (614) of securely transmitting the desired information to an authorized agent. In the embodiments herein, law enforcement officers are provided with the location of the desired vehicle, for example, through encrypted transmission. In some respects, other data that is not the desired information is permanently deleted from the storage area of ​​the processing device.

[0071] Referring here to Figure 7, the method 700 for retrieving and providing desired information is performed within the collection device. Method 700 includes the step (702) of securely receiving at least one rule set database from a processing device that receives information from an authorized agent, uses that information, and updates the database. Method 700 includes the step (704) of securely collecting and storing sensor data associated with pre-selected areas associated with the location of the collection device, and thus, in effect, location tagging the collected data. Method 700 includes the step (706) of filtering the collected data according to “triggers” (rough filtering) from the rule set database. As discussed herein, such filtering steps may include, for example, a step of sorting vehicles from other data, but may include any threshold, a step of sorting vehicle license plates from each other, or even a step of locating specific desired license plate locations. Method 700 includes the steps of securely storing filtered data (708) and securely transmitting the filtered data to a processing device (710). In one aspect, the filtered data is transmitted securely but not stored. Thus, the data is deleted either when they do not meet the trigger criteria, or after or at the time they are securely transmitted to the processing device.

[0072] One or more computer systems may be configured to perform a particular operation or action by having software, firmware, hardware, or a combination thereof that is installed on or causes the system to perform an action when in operation. One or more computer programs may be configured to perform a particular operation or action by including instructions that cause the device to perform an action when executed by a data processing device. One general aspect includes a method for identifying desired information from sensor data collected by a collection device. The method also includes the steps of: securely receiving at least one rule from an authorized agent; determining at least one coarse filter based on at least one rule; updating at least one rule database using at least one rule; securely transmitting at least one coarse filter to a collection device; and securely receiving sensor data from a collection device, wherein the sensor data is filtered by at least one coarse filter. The method also includes the step of determining desired information by applying a fine filter to the filtered sensor data, wherein the fine filter is based on at least one rule database. The method also includes the step of encrypting the desired information and transmitting it to an authorized agent. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0073] An implementation may include one or more of the following features. The method may include the steps of encrypting in place and storing the received roughly filtered sensor data. The rough filter may include at least one feature of interest. At least one feature of interest may include the height of the object. At least one feature of interest may include the model of the vehicle. At least one feature of interest may include the color of the vehicle. The authorized agent may include law enforcement. At least one rule may include, at least in part, a rule generated from an authorization document from the authorized agent. The desired information may include the identification of the object. The desired information may include the license plate number. The method may include the step of securely deleting all roughly filtered sensor data after securely transmitting the desired information to the authorized agent. Implementations of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0074] One general aspect includes a method for identifying desired information from sensor data collected by a collection device. The method also includes the steps of securely receiving at least one rule dataset from a processing device, and securely receiving and storing sensor data associated with a pre-selected area associated with a location of the collection device. The method also includes the step of filtering the sensor data to determine desired information, wherein the filtering step includes at least one step based on a rule database, and the step of securely storing the desired information. The method also includes the step of securely transmitting the desired information to a processing device. Other embodiments of this aspect include a corresponding computer system, an apparatus, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0075] An implementation may include one or more of the following features: The method may include a step of compressing the desired information. The method may include a step of encrypting the desired information. The method may include a step of encoding the desired information. A filtering step may include a step of filtering sensor data and determining a human subject, wherein the filtering step is based on at least one rule database. A filtering step may include a step of filtering sensor data and determining a license plate number, wherein the filtering step is based on at least one rule database. Implementations of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0076] One general aspect includes a system for identifying desired information from sensor data collected by a collection device. The system also includes at least one sensor mounted on an autonomous vehicle, and a collection module running on a processor, the collection module being configured to securely receive at least one rule dataset, the collection module being configured to securely receive and store sensor data associated with a pre-selected area associated with the location of the autonomous vehicle, the collection module being configured to filter based on at least one rule database, the sensor data being used to determine desired information, the collection module being configured to securely store the desired information, and the collection module being configured to securely transmit the desired information. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the Method.

[0077] An implementation may include one or more of the following features: a system in which a collection module is configured to compress the desired information. An implementation of the technique described may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0078] One or more computer systems may be configured to perform a particular operation or action by having software, firmware, hardware, or a combination thereof that is installed on or causes the system to perform an action when in operation. One or more computer programs may be configured to perform a particular operation or action by including instructions that cause the device to perform an action when executed by a data processing device. One general aspect includes a system for identifying desired information from sensor data collected by a collection device, the processing module comprising: a step of securely receiving at least one rule from an authorized agent; a step of updating at least one rule database with at least one rule; a step of securely transmitting at least one rule database to a collection device; a step of securely receiving intermediate coarsely filtered data from a collection device, wherein the intermediate coarsely filtered data is coarsely filtered based on at least one rule database; a step of securely storing the received intermediate coarsely filtered sensor data; and a step of securely storing fine filters of the intermediate data. A processing module includes a processing module computer instruction for the steps of: applying a fine filter to roughly filtered sensor data to determine desired information, wherein the fine filter is based on at least one rule database; securely transmitting the desired information to an authorized agent; and securely deleting all intermediate roughly filtered sensor data; and a collection module configured to run on a collection device, wherein the collection module includes the steps of: securely receiving at least one rule dataset from the processing device executing the processing module computer instruction; securely receiving and storing sensor data associated with a pre-selected area associated with the location of the collection device; and roughly filtering the sensor data.The method comprises a collection module, which includes a collection module computer instruction for determining intermediate roughly filtered sensor data, the step of roughly filtering the data, and at least one rule database, the steps of securely storing the intermediate roughly filtered sensor data and securely transmitting the intermediate roughly filtered sensor data to a processing module; a user interface configured to receive at least one rule from an authorized agent; and a communication gateway configured to enable secure communication between the user interface, the processing module, and the collection module. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0079] An implementation may include one or more of the following features: a system in which a collection module is configured to securely transmit roughly filtered sensor data to a processing module, and then delete all sensor data. Secure communication may include encrypted communication. An implementation of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0080] One general aspect includes a method for identifying desired information from sensor data collected by a collection device. The method also includes the steps of securely receiving at least one rule dataset from a processing device, and securely receiving and storing sensor data associated with a pre-selected area associated with a location of the collection device. The method also includes the step of filtering the sensor data to determine desired information, wherein the filtering step includes at least one step based on a rule database, and the step of securely storing the desired information. The method also includes the step of securely transmitting the desired information to a processing device. Other embodiments of this aspect include a corresponding computer system, an apparatus, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0081] An implementation may include one or more of the following features: The method may include a step of compressing the desired information. The method may include a step of encrypting the desired information. The method may include a step of encoding the desired information. A filtering step may include a step of filtering sensor data and determining a human subject, wherein the filtering step is based on at least one rule database. A filtering step may include a step of filtering sensor data and determining a license plate number, wherein the filtering step is based on at least one rule database. Implementations of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0082] One general aspect includes a system for identifying desired information from sensor data collected by a collection device. The system also includes at least one sensor mounted on an autonomous vehicle, and a collection module running on a processor, the collection module being configured to securely receive at least one rule dataset, the collection module being configured to securely receive and store sensor data associated with a pre-selected area associated with the location of the autonomous vehicle, the collection module being configured to filter based on at least one rule database, the sensor data being used to determine desired information, the collection module being configured to securely store the desired information, and the collection module being configured to securely transmit the desired information. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the Method.

[0083] An implementation may include one or more of the following features: a system in which a collection module is configured to compress the desired information; a collection module is configured to encrypt the desired information; a collection module is configured to encode the desired information; the desired information may include data associated with a human subject; the desired information may include data associated with a license plate. An implementation of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0084] One general aspect is a system for identifying desired information from sensor data collected by a collection device, comprising a processing module, the steps of: securely receiving at least one rule from an authorized agent; updating at least one rule database using at least one rule; securely transmitting at least one rule database to a collection device; securely receiving intermediate roughly filtered data from a collection device, wherein the intermediate roughly filtered data is roughly filtered based on at least one rule database; securely storing the received intermediate roughly filtered sensor data; applying a fine filter to the securely stored intermediate roughly filtered sensor data to determine the desired information, wherein the fine filter is based on at least one rule database; securely transmitting the desired information to an authorized agent; and all intermediate roughly filtered A processing module, comprising a processing module computer instruction for the step of securely deleting filtered sensor data; a collection module configured to run on a collection device, wherein the collection module comprises a collection module computer instruction for the steps of securely receiving at least one rule dataset from a processing device executing the processing module computer instruction; securely receiving and storing sensor data associated with a pre-selected area associated with a location of the collection device; and roughly filtering the sensor data and determining intermediate roughly filtered sensor data, wherein the rough filtering step is based on at least one rule database; securely storing the intermediate roughly filtered sensor data; and securely transmitting the intermediate roughly filtered sensor data to the processing module; and a user interface configured to receive at least one rule from an authorized agent.It includes a communication gateway configured to enable secure communication between the user interface, processing modules, and collection modules. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of this method.

[0085] An implementation may include one or more of the following features: A system which may include data associated with a human subject for desired information; the desired information may include data associated with a license plate; the collection module is configured to securely transmit coarsely filtered sensor data to a processing module and then delete all sensor data; secure communication may include encrypted communication; the collection module is configured to encode the desired information; a method which includes at least one rule which may include: the method which includes a rule generated at least in part from an authorization from an authorized agent, the authorized agent being a law enforcement agency; the method further includes: the method which includes at least a step of determining the identification of a subject based on the results from a fine filter; the method further includes: the method which includes at least a step of determining a license plate number based on the results from a fine filter; the method further includes: the method which includes securely transmitting the desired information to an authorized agent and then securely deleting all intermediate coarsely filtered sensor data. Implementations of the techniques described may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0086] An implementation may include one or more of the following features: A method in which at least one rule may include: A method which includes, at least in part, a rule generated from an authorization from an authorized agent, the authorized agent being a law enforcement agency; The method further includes: A method which includes at least a step of determining the identification of a subject based on the results from a fine-grained filter; The method further includes: A method which includes at least a step of determining a license plate number based on the results from a fine-grained filter; The method further includes: A method which includes securely deleting all intermediate, coarsely filtered sensor data after securely transmitting the desired information to an authorized agent. Implementations of the described techniques may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0087] In various aspects, the system may enable law enforcement agencies or relevant authorities using the system to instruct the system on what it should search for. The system may patrol actively, and to the extent authorized by law enforcement, law enforcement may instruct the system to look for specific things, such as a particular face, sound, vehicle model, license plate, etc. The system may also be instructed to include items such as location and / or time. The system may also be configured to refer to the absence of an item in the detected scene that would normally be present. In a non-limiting embodiment, the system may detect the absence of trucks in a parking area where five trucks are assumed to be parked, but only four trucks are parked. The system's collection device may, in some configurations, be juxtaposed with, housed in or on, or performed by an autonomous vehicle. A non-limiting embodiment of a preferred autonomous vehicle is described in U.S. Patent Application No. 16 / 435,007 (Patent Attorney No. AA001), filed on 7 June 2019, entitled "System and Method for Distributed Utility Service Execution" (which is fully incorporated herein by reference). The acquisition module may perform relatively coarse-grained filtering on the acquired data to transmit only the desired data so that unwanted data is not transmitted. The acquisition module may be configured to downsample the data in real time as the data is acquired, for example, within a set of acquired images that depict the same item (and add little or no information to each other such that each differs from its nearest neighbor by a threshold percentage or absolute delta), so that only one or a subset of such acquired images is transmitted. In addition, the acquisition module may be configured to compress the data to be transmitted in order to conserve bandwidth.The collection module may be configured to transmit a window of images around the matched image, starting with images detected at a time before the matched image is detected and ending with images detected at a time after the matched image is detected, depending on the step of detecting a matched image for a desired object or item. The collection module may be configured to encrypt all data that will be transmitted. Fine-grained filtering may be performed in the processing module to which the collection module has transmitted the data. In a non-limiting embodiment, the processing module may be located in the cloud. The relative coarseness and relative fineness of the filtering may be adjusted relative to each other as needed, for example, for system optimization. In addition, or alternatively, the relative coarseness of the filtering performed in the collection module may be adjusted at least in part based on the type of object desired and / or the privacy expectation associated with the object desired. Some types of objects, e.g., certain faces, may be more difficult to detect reliably than other types of objects, e.g., license plates. For example, some types of objects, e.g., certain faces, may be associated with a higher privacy expectation compared to other types of objects, e.g., license plates.

[0088] In at least one array, a processing module may create a rule set. An authorized agent may be authenticated to a web-based application that enables authorized agents to create new rule sets. In addition, or alternatively, a trusted external system may create new rule sets via a Representational State Transfer (REST) ​​Application Programming Interface (API). In at least one array, a rule set may include at least the following rule information: (i) a target / rule identifier that associates the rule and any results with the agent or with a system of the agent's records (e.g., authorization number, case number, etc.); (ii) a target type (e.g., person, vehicle license plate, etc.); (iii) a target compared (e.g., person image, license plate number); (iv) a search valid date / time; (v) a search expiration date / time; and (vi) a search location. A web-based application may validate the information provided and create a filter object based on the target type and the target input being compared. In some aspects, a person filter object may not contain an image of the target. The collection module may detect any / all people, not just specific individuals; therefore, the face recognition module may reside in the cloud, and thus, target images do not need to be sent to the collection module. In some aspects, the license plate filter object may contain an alphanumeric string. In some aspects, the information in the rule set about the target type may be in the form of a JavaScript® Object Notation (JSON) object.

[0089] In at least one array, the processing module may assign one or more rule sets to one or more autonomous devices, e.g., robots, or bots. Bots may incorporate data collection modules. A web-based application may select a list of active bots located within search locations defined in a database of current bot locations. In some aspects, the system may encrypt filter packages using the AES encryption algorithm at 256-bit strength. In addition, or alternatively, the web-based application may transmit filter packages to selected bots using Transport Layer Security (TLS) 1.2.

[0090] In at least one array, the collection module may collect images of detected objects that match the target type of the rule set (e.g., person, license plate number). In some aspects, the system may broadly filter objects based on the rule set. For example, if the rule set defines that an object of interest has license plate number x, the system may filter out only vehicles (cars, trucks, motorcycles, etc.) among all detected objects. In some aspects, the system may provide a two-dimensional (2D) bounding box tracking mechanism, such as generalized union crossing (GIOU) tracking, to track people or vehicles and their individual bounding boxes from one image frame to the next. In some aspects, as a non-limiting embodiment, the system may crop some or all segments of the image within the bounding box of a person. The system may send at least one such cropped image associated with each detected and tracked object whose type is person or license plate number to a processing module. In some aspects, the system may encode and / or encrypt the cropped image data before sending it to a processing module.

[0091] In at least one array, the processing module may receive and process cropped images of objects to be detected and tracked within the location and time defined by the rule set. In some aspects, the system may decrypt and / or decode cropped images generated by the collection module and provided to the processing module. The processing steps may continue if the data is valid, i.e., the data can be decrypted and decoded. Invalid data, i.e., data that cannot be decrypted or decoded, may be deleted. In some aspects, the system may verify that the timestamp and / or GPS location of the cropped images are within the boundaries indicated by the search authorization, delete invalid data, and store valid data for further processing.

[0092] In at least one array, the processing module may receive and process cropped images of license plates detected and tracked within a time and place defined by a rule set. In some aspects, the system may decrypt and / or decode cropped images generated by the collection module and provided to the processing module. The system may provide at least one decoded and cropped image as input to a neural network algorithm that can generate at least one bounding box containing at least one license plate as output. In some aspects, the system may further crop detected license plates from each input to the neural network algorithm for further processing. In some aspects, the cropped images containing license plates may be subjected to an optical feature recognition algorithm to output alphanumeric characters on the license plate as text. In some aspects, the system may perform text matching between the generated text and license plate numbers described in a filter rule set to classify the text as a match or a non-match.

[0093] In at least one array, the processing module may receive and process cropped images of a target, which are detected and tracked within a time and place defined by a rule set. In some aspects, the system may decrypt and / or decode cropped images generated by the collection module and provided to the processing module. The system may provide at least one decoded and cropped image as input to a neural network algorithm that can generate at least one bounding box containing at least one face image as output. In some aspects, the system may further crop detected face images from each input to the neural network algorithm for further processing.

[0094] In at least one array, the system can encrypt and store each cropped image containing the matched text or the matched image.

[0095] In at least one array, an authorized agent may view information about the matched target or matching event. The processing module may notify the authorized agent about the matched target. In some aspects, the processing module may create and store encrypted packages containing GPS coordinates and / or timestamps and / or images and / or text associated with the matched target. The system may notify an authorized agent by sending an SMS message and / or an email message, or a message containing a URL link to a web-based application. In addition, or alternatively, the system may notify authorization by sending the notification to a trusted third-party web service via a message queue.

[0096] In at least one array, an authorized agent may view and / or download and / or ingest the matched target data. The agent may authenticate to a web-based application, thereby viewing and / or downloading the matched target data. In addition, or alternatively, the processing module may send an access token to a trusted third party for use in reading encrypted packages.

[0097] In at least one array, face recognition and / or text recognition modules may be trained periodically to maintain a thresholded prediction accuracy over time. In some aspects, the prediction confidence threshold is initially set to a relatively low level that generates more misdeterminants, and is increased over time as the model becomes more accurate, generating fewer overall matches.

[0098] In at least one array, for example, a confidence level threshold for filtering data to determine whether to validate, reserve, invalidate, or discard the data may be set to 85%. In some aspects, any data that cannot be decrypted or decoded may be discarded. In some aspects, to that effect, in a non-limiting embodiment, any data that a bot cannot authenticate may be discarded. In some aspects, for example, any data outside the scope of the authorization may be discarded.

[0099] In at least one configuration, security measures may be maintained between the bot with the collection module and the bot's remote controller operator. In a non-limiting embodiment, the remote control operator may be required to authenticate using a username and password to access the remote control console. In some aspects, the remote control console may include a web browser that can transmit connection requests, which may be encrypted using the AES encryption algorithm at 256-bit strength, and which may be transmitted using Transport Layer Security (TLS) 1.2. In some aspects, the identification of the remote control console and the bot may be authenticated by a connection broker prior to the creation of a peer-to-peer connection. In a non-limiting embodiment, the web browser of the remote control console may establish a secure peer-to-peer connection with the bot via WebRTC. In some aspects, video from the remote control console display may be streamed to the browser via WebRTC and may not be stored within the web browser or on the remote console or remote console host machine.

[0100] Those skilled in the art will understand that the methods described herein may be applied to computer systems configured to perform such methods, and / or computer-readable media containing programs for performing such methods, and / or software and / or firmware and / or hardware (e.g., integrated circuits) designed to perform such methods. Raw data and / or results may be stored, printed, displayed, transferred to another computer, and / or transferred to any location for future reading and processing. Communication links may be wired or wireless, including, in non-limiting embodiments, Ethernet®, cellular or wideband networks, WiFi or local area networks, military communication systems, and / or satellite communication systems. Parts of the system may operate, for example, on a computer having a variable number of CPUs. Other alternative computer platforms may be used.

[0101] As those skilled in the art will understand, the methods described herein can be implemented electronically, in whole or in part. Signals representing actions taken by elements of the systems of the disclosed and other disclosed configurations can be transmitted over at least one live communication network. Control and data information can be executed electronically and stored on at least one computer-readable medium. The system can be implemented to run on at least one computer node in at least one live communication network. Common forms of computer-readable medium include, but are not limited to, floppy disks, flexible disks, hard disks, magnetic tape or any other magnetic medium, compact disk read-only memory or any other optical medium, punch cards, paper tape or any other physical medium with a pattern of holes, random access memory, programmable read-only memory, erasable programmable read-only memory (EPROM), flash EPROM or any other memory chip or cartridge, or any other medium from which a computer can read.

[0102] Those skilled in the art will understand that information and signals can be represented using any of a variety of existing techniques. For example, data, instructions, commands, information, signals, bits, symbols, or chips that may be referred to throughout this description may be represented by voltage, current, electromagnetic waves, magnetic fields or particles, optical fields or particles, ultrasound, projected capacitance, or any combination thereof.

[0103] Those skilled in the art will further understand that various illustrative logic blocks, modules, circuits, and algorithmic steps described in relation to the arrays disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this hardware and software compatibility, various illustrative components, blocks, modules, circuits, and steps are described in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each specific application, but such a decision on implementation should not be construed as causing a departure from the scope of the appended claims.

[0104] Various illustrative logic blocks, modules, and circuits described in connection with the arrays disclosed herein may be implemented, or implemented using, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0105] Actions of methods or algorithms described in relation to the sequences disclosed herein may be embodied directly in hardware, in software modules executed by a processor, or in a combination of both. The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. The storage medium may be coupled to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium may be integrated with the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a functional device such as a computer, robot, user terminal, mobile phone or tablet, car, or IP camera. Alternatively, the processor and storage medium may reside as discrete components within such a functional device.

[0106] The above description is not intended to be exhaustive or to limit the features to the precise forms disclosed. Various alternatives and modifications can be devised by those skilled in the art without departing from the disclosure, and the general principles defined herein may be applied to other aspects without departing from the spirit or scope of the appended claims. Thus, the disclosure is intended to encompass all such alternatives, modifications, and differences. In addition, while several arrays of the disclosure are shown in the drawings and / or discussed herein, the disclosure is not intended to be limited thereto, as it is intended to be as broad as what the art will enable and as equally important to read. Accordingly, the above description should not be construed as restrictive, but merely as an embodiment of a particular configuration. Furthermore, those skilled in the art will recall other modifications within the scope and spirit of the claims appended to the specification. Other elements, steps, actions, methods, and techniques that are not substantially different from those described above and / or in the appended claims are also intended to be within the scope of the disclosure. Accordingly, the appended claims are not intended to be limited to the sequences shown and described herein, but are intended to be of the broadest possible scope that is consistent with the principles and novel features disclosed herein.

[0107] The arrangements shown in the drawings are presented solely to demonstrate certain embodiments of the present disclosure. Furthermore, the drawings described are merely illustrative and non-limiting. In the drawings, for illustrative purposes, some of the dimensions of elements may be exaggerated and not depicted to a particular scale. In addition, elements shown in the drawings with the same number may, depending on the context, be the same element or similar elements.

[0108] When the term “comprising” is used in this description and claims, it does not exclude other elements or steps. When an indefinite or definite article, such as “a,” “an,” or “the,” is used to refer to a singular noun, it also includes the plural form of that noun unless otherwise specifically stated. Thus, the term “comprising” should not be interpreted as being limited to the items listed thereafter, and it does not exclude other elements or steps, and therefore the scope of the expression “a device comprising item A and B” should not be limited to a device consisting only of components A and B. Furthermore, to the extent that the terms “includes,” “has,” “possesses,” and their equivalents are used in this description and claims, such terms are intended to be inclusive in a form similar to “comprising” so that when adopted as transitional terms in a claim, they are interpreted as “comprising.”

[0109] Furthermore, the terms “first,” “second,” “third,” and their equivalents are provided to distinguish between similar elements, whether used within this description or in the claims, and not necessarily to describe order or chronology. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances (unless otherwise expressly disclosed), and that embodiments of the disclosures described herein may operate in sequences and / or arrangements other than those described or illustrated herein.

[0110] One or more computer systems may be configured to perform a particular operation or action by having software, firmware, hardware, or a combination thereof that is installed on or causes the system to perform an action when in operation. One or more computer programs may be configured to perform a particular operation or action by including instructions that cause the device to perform an action when executed by a data processing device. One general aspect includes a method for identifying desired information from sensor data collected by a collection device. The method also includes the steps of: securely receiving at least one rule from an authorized agent; updating at least one rule database with at least one rule; securely transmitting at least one rule database to a collection device; securely receiving sensor data from a collection device, wherein the sensor data is coarsely filtered based on at least one rule database; and securely storing the received coarsely filtered sensor data. The method also includes the step of applying a fine filter to the securely stored and filtered sensor data to determine desired information, wherein the fine filter is based on at least one rule database. The method also includes the step of securely transmitting the desired information to an authorized agent. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0111] An implementation may include one or more of the following features: at least one rule may include, at least in part, a rule generated from an authorization from an authorized agent, where the authorized agent is a law enforcement agency; the method may include at least a step of determining the identification of a subject based on the results from a fine-grained filter; the method may include at least a step of determining a license plate number based on the results from a fine-grained filter; the method may include a step of securely deleting all intermediate, coarsely filtered sensor data after securely transmitting the desired information to an authorized agent. Implementations of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0112] One general aspect includes a method for identifying desired information from sensor data collected by a collection device. The method also includes the steps of securely receiving at least one rule dataset from a processing device, and securely receiving and storing sensor data associated with a pre-selected area associated with a location of the collection device. The method also includes the step of filtering the sensor data to determine desired information, wherein the filtering step includes at least one step based on a rule database, and the step of securely storing the desired information. The method also includes the step of securely transmitting the desired information to a processing device. Other embodiments of this aspect include a corresponding computer system, an apparatus, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the method.

[0113] An implementation may include one or more of the following features: The method may include a step of compressing the desired information. The method may include a step of encrypting the desired information. The method may include a step of encoding the desired information. A filtering step may include a step of filtering sensor data and determining a human subject, wherein the filtering step is based on at least one rule database. A filtering step may include a step of filtering sensor data and determining a license plate number, wherein the filtering step is based on at least one rule database. Implementations of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0114] One general aspect includes a system for identifying desired information from sensor data collected by a collection device. The system also includes at least one sensor mounted on an autonomous vehicle, and a collection module running on a processor, the collection module being configured to securely receive at least one rule dataset, the collection module being configured to securely receive and store sensor data associated with a pre-selected area associated with the location of the autonomous vehicle, the collection module being configured to filter based on at least one rule database, the sensor data being used to determine desired information, the collection module being configured to securely store the desired information, and the collection module being configured to securely transmit the desired information. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of the Method.

[0115] An implementation may include one or more of the following features: a system in which a collection module is configured to compress the desired information; a collection module is configured to encrypt the desired information; a collection module is configured to encode the desired information; the desired information may include data associated with a human subject; the desired information may include data associated with a license plate. An implementation of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0116] One general aspect is a system for identifying desired information from sensor data collected by a collection device, comprising a processing module, the steps of: securely receiving at least one rule from an authorized agent; updating at least one rule database using at least one rule; securely transmitting at least one rule database to a collection device; securely receiving intermediate roughly filtered data from a collection device, wherein the intermediate roughly filtered data is roughly filtered based on at least one rule database; securely storing the received intermediate roughly filtered sensor data; applying a fine filter to the securely stored intermediate roughly filtered sensor data to determine the desired information, wherein the fine filter is based on at least one rule database; securely transmitting the desired information to an authorized agent; and all intermediate roughly filtered A processing module, comprising a processing module computer instruction for the step of securely deleting filtered sensor data; a collection module configured to run on a collection device, wherein the collection module comprises a collection module computer instruction for the steps of securely receiving at least one rule dataset from a processing device executing the processing module computer instruction; securely receiving and storing sensor data associated with a pre-selected area associated with a location of the collection device; and roughly filtering the sensor data and determining intermediate roughly filtered sensor data, wherein the rough filtering step is based on at least one rule database; securely storing the intermediate roughly filtered sensor data; and securely transmitting the intermediate roughly filtered sensor data to the processing module; and a user interface configured to receive at least one rule from an authorized agent.It includes a communication gateway configured to enable secure communication between the user interface, processing modules, and collection modules. Other embodiments of this aspect include a corresponding computer system, a device, and a computer program recorded on one or more computer storage devices, each configured to perform the actions of this method.

[0117] An implementation may include one or more of the following features: a system in which a collection module is configured to securely transmit roughly filtered sensor data to a processing module, and then delete all sensor data. An implementation of the described technique may include hardware, methods or processes, or computer software on a computer-accessible medium.

[0118] The charges will be as follows:

Claims

1. A method for identifying desired information from sensor data collected by a device, wherein the method is: Securely receiving rules from authorized agents, Based on the aforementioned rules, determine the first filter, The rule database is updated using the aforementioned rules, The first filter is securely transmitted to the collection device, The sensor data is securely received from the collection device, and the sensor data received in real time is filtered by the first filter. The desired information is determined by applying a second filter to the filtered sensor data, wherein the application of the second filter to the filtered sensor data is performed remotely from the filtering of the sensor data by the first filter, based on the rule database. Encrypt the aforementioned desired information and transmit it to the authorized agent. Methods that include...

2. The method according to claim 1, further comprising encrypting the sensor data by the collection device and storing the sensor data filtered by the received first filter.

3. The aforementioned authorized agent, Law enforcement agencies The method according to claim 1, including the method described in claim 1.

4. The rule is, Rules generated, at least in part, from authorization documents from the authorized actors. The method according to claim 1, including the method described in claim 1.

5. The desired information is Identification of the target The method according to claim 1, including the method described in claim 1.

6. The desired information is License plate number The method according to claim 1, including the method described in claim 1.

7. After securely transmitting the desired information to the authorized agent, securely delete all sensor data filtered by the first filter. The method according to claim 1, further comprising:

8. The method according to claim 1, wherein the sensor data is digitally signed by the collection device.

9. The method according to claim 1, wherein the first filter is a coarse filter and the second filter is a fine filter.

Images