Systems and methods for object recognition using neural networks

Abstract

A system and method for providing object recognition using an artificial neural network. The method includes capturing a plurality of reference images using a camera associated with an edge node on a communication network. The reference images are received by a centralized server on the communication network. The reference images are analyzed using a parent neural network of the centralized server to determine a subset of objects in the reference images that are recognized by the parent neural network. One or more filters responsive to the subset of objects are selected from the parent neural network. A pruned neural network is created based only on the one or more filters. The pruned neural network is deployed to the edge node. Real-time images are captured using the camera of the edge node and objects in the real-time images are recognized using the pruned neural network.

Description

Technical Field

[0001] The present disclosure relates generally to object recognition systems, and more particularly to object recognition systems using artificial neural networks. Background Art

[0002] Object classification or recognition for captured images and videos is becoming an increasingly important capability for various intelligent systems. Prior art image and video processing for computer-assisted object recognition is implemented using deep learning artificial neural networks (ANNs). In particular, convolutional neural networks (CNNs) have shown great promise in computer-assisted object recognition applications.

[0003] According to the current state of the art in object recognition systems, deep learning models built using convolutional layers (i.e., CNNs) are able to recognize approximately a thousand or more different object categories with near-perfect accuracy. Despite their outstanding performance in accurately performing the above techniques, achieving such performance levels feasibly requires a vast amount of computing resources. For this reason, CNNs and other neural networks cannot be extended to resource-constrained computing devices, such as consumer electronics, without making significant compromises in performance (in terms of speed and accuracy).

[0004] Therefore, there is a continuing need in the art for an image processing computer system that can accurately classify objects using less computing resources. Summary of the invention

[0005] The present disclosure relates to an inventive system and method for improving pedestrian safety and / or performing surveillance, particularly at resource-constrained edge nodes in a communication network, via object recognition using a pruned neural network. The pruned neural network is created by a centralized server of the communication network so as to be customized for each of the edge nodes on the server. The creation of the pruned neural network includes identifying the subject matter of the objects (or categories) of the parent neural network and selecting filters that are highly responsive to those objects. A subset of objects is determined by analyzing a reference image actually captured at each edge node, such that the pruned neural network is specifically customized for each specific edge node. In addition, the analysis of the reference image is performed by the parent neural network, which is unpruned, fully trained, and comprehensive. Ultimately, this enables the creation of smaller neural networks filter by filter based on the results of the parent neural network.

[0006] Since the parent neural network is used to provide image recognition and enable filter selection for objects in the reference image, the pruned neural network has minimal decrease in accuracy, but significant gains in reduced speed of inference and computational requirements. By deploying a pruned neural network for each edge node that includes only a subset of objects relevant to each specific edge node, less powerful or resource-constrained computing devices (e.g., consumer electronics) can quickly detect certain relevant objects with very high accuracy using minimal resources.

[0007] In general, in one aspect, a method for recognizing an object using a neural network is provided. The method includes the steps of capturing a plurality of reference images using a camera associated with an edge node on a communication network; receiving the reference images using a centralized server on the communication network; analyzing the reference images using a parent neural network of the centralized server to determine a subset of objects in the reference images that are recognized by the parent neural network; selecting one or more filters in the parent neural network that are responsive to the subset of objects; creating a pruned neural network based on the one or more filters generated by the selecting step; deploying the pruned neural network to the edge node; capturing a real-time image using the camera of the edge node; and recognizing an object in the real-time image using the pruned neural network.

[0008] According to one embodiment, the method is performed for each of a plurality of edge nodes each connected to the communication network. According to one embodiment, the cameras of the first edge node and the second edge node are configured to observe a common geographic area, the method further comprising dividing the subset of objects into a first portion and a second portion, such that the pruned neural network of the first edge node is created from those filters corresponding to the first portion, and the pruned neural network of the second edge node is created from those filters corresponding to the second portion.

[0009] According to one embodiment, each of the edge nodes comprises a light emitter of a networked lighting system. According to a further embodiment, the networked lighting system comprises one or more lamp posts, street lights or traffic lights, and wherein the method is used to improve pedestrian safety.

[0010] According to one embodiment, the parent neural network and the pruned neural network are convolutional neural networks (CNNs). According to one embodiment, the selecting step includes analyzing a plurality of known training images (68) using the parent neural network. According to one embodiment, selecting one or more filters includes selecting filters that exhibit a maximum response to the known training images. According to one embodiment, wherein the plurality of reference images are formed from one or more videos. According to one embodiment, the analyzing includes creating a frequency map (62) by counting the number of times each object category is recognized by the parent neural network, and determining the object subset based on the frequency map.

[0011] In general, in one aspect, an object recognition system for a communication network having a plurality of edge nodes is provided. The system includes a centralized server for the communication network, the centralized server having: a parent neural network configured to analyze a reference image received by the centralized server from each of the edge nodes to determine a subset of objects in the reference image corresponding to each of the edge nodes; and a pruning application configured to select one or more filters in the parent neural network that are responsive to each subset of objects, and to create a pruned neural network for each edge node based on the one or more filters corresponding to each particular edge node.

[0012] According to one embodiment, the edge node comprises one or more light emitters. According to one embodiment, the one or more light emitters comprise one or more lamp posts, street lights, or traffic lights. According to one embodiment, the system is implemented to enhance pedestrian safety. According to one embodiment, the centralized server comprises a memory and a processor, and the parent neural network and the pruning application are installed in the memory and executed by the processor.

[0013] By "edge node" is meant a device that is not part of the centralized infrastructure of a network but is connected to the network and may or may not assist in the functionality of the network. Examples of edge nodes include smartphones, laptops, consumer electronics or appliances, such as smart / connected management systems for lighting, thermostats, security or other appliances, cellular or web-enabled cars, etc., or accessories for use with them.

[0014] As used herein, an "artificial neural network," "ANN," or simply "neural network" includes a computer system that includes software and / or hardware components according to any known or discovered paradigm that gives the neural network the ability to learn (progressively improve performance based on inputs of known examples), as opposed to software applications that are programmed in a task-specific manner. ANNs are typically arranged as interconnected groups of "neurons" that generally simulate the deployment and workings of the human brain. The "neurons" are arranged in layers that include filters that respond differently depending on the data input to each neuron. Training of the ANN can be accomplished by inputting known examples into the ANN, which enables the ANN to adjust the weights of different neurons to more closely correspond to the known examples. In this way, the ANN is able to accurately process unknown inputs by comparing which known examples the unknown inputs are most similar to based on the filters that respond.

[0015] Convolutional neural networks (CNNs) are a type of ANN that shows particular promise when used with the embodiments disclosed herein. CNNs are typically constructed as stacks of different layers that ultimately transform inputs (e.g., images) into outputs (e.g., the categories of (multiple) objects that the CNN recognizes in the image). Convolutional layers are the core building blocks of CNNs and include a collection of learnable filters. When presented with an image, different filters in these filters will be activated in response to different patterns in the image, which in turn correspond to different objects. The basis for decisions made in such a neural network is measured by the response of the activated layers, that is, which filters are activated. Based on the responses of the filters, the classification layers of CNNs typically assign scores corresponding to the probability likelihood of which (multiple) objects in the image belong to which (which) categories. The Softmax function is an example of a function that can be used by the classification layer of a CNN, where a larger result from the function is associated with an increased likelihood that the (multiple) objects in the image have been correctly classified.

[0016] An advantageous property of some ANNs, such as CNNs, is their ability to learn filter weights associated with perceived representations of objects, information that can be sent to higher-level layers (closer to the output) in the ANN to refine the ANN's understanding of the object. The ability of the ANN to learn abstract representations of objects can be further improved by adding additional filters and layers. However, the addition of filters and layers results in an increase in computational complexity and a corresponding decrease in performance (e.g., the time required to analyze an image and / or the accuracy of identifying / classifying objects). Powerful machines including one or more designated processors, such as graphics processing units (GPUs), are typically used to implement the processing required for CNNs. The edge nodes of edge-based computing systems are devices that may not include GPUs and / or have relatively low amounts of computing resources, and therefore cannot be feasibly used to provide the processing requirements of a comprehensive CNN.

[0017] In various embodiments disclosed herein, an edge node includes a networked lighting system having multiple light emitting facilities or light emitters connected together in a communication network. The terms "light emitting facility" or "light emitter" are used herein to refer to an implementation or arrangement of one or more light emitting units of a specific form factor, assembly or packaging. The term "light emitting unit" is used herein to refer to a device that includes one or more light sources of the same or different types. A given light emitting unit can have any of a variety of mounting arrangements for (multiple) light sources, housing / housing arrangements and shapes, and / or electrical and mechanical connection configurations. In addition, a given light emitting unit can optionally be associated with (e.g., include, be coupled to, and / or be packaged together with) various other components (e.g., control circuits) related to the operation of (multiple) light sources.

[0018] The term "light source" should be understood to refer to any one or more of a variety of radiation sources, including but not limited to LED-based sources (including one or more LEDs as defined above), incandescent sources (e.g., filament lamps, halogen lamps), fluorescent sources, phosphor sources, high intensity discharge sources (e.g., sodium vapor, mercury vapor and metal halide lamps), lasers, other types of electroluminescent sources, pyroluminescent sources (e.g., flames), candlelight sources (e.g., gas lampshades, carbon arc radiation sources), photoluminescent sources (e.g., gas discharge sources), cathode luminescent sources using electron saturation, current luminescent sources, crystal luminescent sources, cathode luminescent sources, thermoluminescent sources, triboluminescent sources, sonoluminescent sources, radioluminescent sources and light-emitting polymers, but not limited to these.

[0019] As used herein, the term "network" refers to any interconnection of two or more devices (including controllers or processors) that facilitates information transmission (e.g., for device control, data storage, data exchange, etc.) between any two or more devices and / or between multiple devices coupled to the network. As should be readily appreciated, various embodiments of a network suitable for interconnecting multiple devices may include any of various network topologies and employ any of various communication protocols. In addition, in various networks according to the present disclosure, any connection between two devices may represent a dedicated connection between two systems, or may alternatively represent a non-dedicated connection. In addition to carrying information intended for two devices, such a non-dedicated connection may carry information that is not necessarily intended for any of the two devices (e.g., an open network connection). In addition, it should be readily appreciated that various device networks as discussed herein may employ one or more wireless, wired / cable, and / or optical fiber links to facilitate information transmission throughout the network.

[0020] In various embodiments, a processor or controller may be associated with one or more storage media (generally referred to herein as "memory", e.g., volatile and non-volatile computer memory, such as RAM, PROM, EPROM and EEPROM, floppy disks, compact disks, optical disks, magnetic tapes, etc.). In some embodiments, the storage media may be encoded with one or more programs that, when executed on one or more processors and / or controllers, implement at least some of the functions discussed herein. The various storage media may be fixed within the processor or controller or may be transferable so that the one or more programs stored therein can be loaded into the processor or controller in order to implement the various aspects of the invention discussed herein. The term "program" or "computer program" is used herein in a general sense to refer to any type of computer code (e.g., software or microcode) that can be used to program one or more processors or controllers.

[0021] It should be appreciated that all combinations of the above concepts and additional concepts discussed in more detail below (assuming such concepts are not mutually inconsistent) are contemplated as part of the inventive subject matter disclosed herein. In particular, all combinations of the claimed subject matter appearing at the end of this disclosure are contemplated as part of the inventive subject matter disclosed herein. It should also be appreciated that terminology that is expressly adopted herein and that may also appear in any disclosure incorporated by reference should be given the meaning most consistent with the specific concepts disclosed herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.

[0023] Figure 1 An object recognition system according to one embodiment disclosed herein is schematically illustrated.

[0024] Figure 2 An object recognition system arranged in an example scenario according to one embodiment disclosed herein is schematically illustrated.

[0025] Figure 3 is training and operating such as Figure 1 Flow chart of a method of an object recognition system of a system.

[0026] Figure 4 is a flow chart further describing a pruning routine for creating a pruned neural network according to one embodiment disclosed herein. DETAILED DESCRIPTION

[0027] The present disclosure describes various embodiments of an object recognition system with multiple edge nodes. More generally, applicants have recognized and appreciated that it would be advantageous to provide a centralized server with a pruning application for creating a customized pruned neural network for each edge node based on the recognized objects in the images actually captured by each edge node. A particular goal of certain embodiments of the present disclosure is to enable accurate and real-time image recognition using neural networks on edge nodes with limited computing resources.

[0028] In view of the above, various embodiments and implementations are directed to systems and methods for improving pedestrian safety and / or enforcing regulation, particularly at resource-constrained edge nodes in a communication network, via object recognition performed using pruned neural networks. The pruned neural network is created by a centralized server of the communication network so as to be customized for each edge node on the server. The pruned neural network is created by a centralized server of the communication network so as to be customized for each of the edge nodes on the server. The creation of the pruned neural network includes identifying the subject matter of the objects (or categories) of the parent neural network and selecting filters that are highly responsive to those objects. A subset of objects is determined by analyzing a reference image actually captured at each edge node, such that the pruned neural network is specifically customized for each specific edge node. In addition, the analysis of the reference image is performed by a parent neural network, which is unpruned, fully trained, and comprehensive. Ultimately, this enables the creation of smaller neural networks filter by filter based on the results of the parent neural network.

[0029] Since the parent neural network is used to provide image recognition and enable filter selection for objects in the reference image, the pruned neural network has minimal loss in accuracy, but significant gains in reduced speed of inference and computational requirements. By deploying a pruned neural network for each edge node that includes only a subset of objects relevant to each particular edge node, less powerful or resource-constrained computing devices (e.g., consumer electronics) can quickly detect certain objects of interest with very high accuracy using minimal resources.

[0030] refer to Figure 1 In one embodiment, the object recognition system 10 is provided with a plurality of edge nodes 12 that communicate with a centralized server 14 that is part of a communication network 15 (or simply "network 15") represented by a dashed line. The network 15 may include any desired wired and / or wireless communication technology, protocol, or standard, including those commercially available under the names of Bluetooth, Wi-Fi, ZigBee, LTE, etc., or combinations thereof. The edge nodes 12 may be directly connected to the centralized server 14, or indirectly connected through one or more other nodes (not shown), particularly where the communication network 15 is embodied as a wireless mesh network.

[0031] Each node 12 takes the general form of a computerized device that includes a processor 16, a memory 18, and a network interface 20. The processor 16 may take the form of any electronic circuit, control unit, and / or logic unit that is configured to perform operations specified by software instructions transmitted to the processor 16. The processor 16 may be or include any central processing unit (CPU) or graphics processing unit (GPU) known or developed in the computing industry. The software (e.g., applications, programs, programming codes, etc.) executed by the processor 16 and the data generated by the operations performed by the processor or used during the operations may be stored in the memory 18. The network interface 20 may include a transmitter, a receiver, a radio, or any other hardware or software component required to enable the edge node 12 to communicate on the network 15.

[0032] Although only three edge nodes 12 are illustrated, it is to be understood that any number of three edge nodes 12 may be included. Communication between the edge nodes 12 and the centralized server 14 may be accomplished directly, or indirectly via one or more intermediate nodes (edge ​​nodes or otherwise) that operate to forward or relay communications between the edge nodes 12 and the centralized server 14.

[0033] Each node 12 is provided with a camera 22. The camera 22 may be or include any device capable of capturing one or more images - still shots or video. The functionality of each camera 22 may be supported by a corresponding processor 16 and images stored in a corresponding memory 18. For example, the processors 16 may be arranged to cause their respective cameras 22 to take pictures at regular intervals (e.g., once every minute or other time units), take pictures in response to a sensed trigger condition or event (e.g., detection of movement), or take pictures in any other desired manner.

[0034] Each edge node 12 is also provided with an artificial neural network 24 (interchangeably, "ANN" or simply "neural network"). The neural network 24 may include software and / or hardware components according to any known or disclosed paradigm. In particular, the neural network applicable to the embodiments disclosed herein is configured to analyze an image (e.g., captured by the camera 22), detect objects in the image, and classify them. To this end, the ANN 24 may include software installed on the memory 18 that can be executed by the processor 16. In one embodiment, the ANN 24 takes the form of a convolutional neural network (CNN) because CNNs have shown particularly outstanding results when used for image recognition. Examples of suitable CNNs include AlexNet, ImageNet, Microsoft Common Objects in Context (COCO), etc., but other examples may also be used.

[0035] In one embodiment, the system 10 is or includes a networked or "smart" lighting system that includes nodes 12 that are multiple light emitters that are networked together. Because the light emitters of the lighting system are not always actively computing (e.g., the light emitters are typically idle when not receiving or implementing specific instructions), the networked lighting system provides idle computing resources that can be put to use by the ANN 24. Moreover, the networked lighting system is advantageously installed or can be installed in a variety of locations that can benefit from monitoring or surveillance, such as in buildings, rooms, urban landscapes, roads, traffic intersections, parks, fields, indoors, outdoors, and the like.

[0036] In one embodiment, the system 10 is arranged to monitor a location to provide improved pedestrian safety. For example, the system 10 can be used to detect the presence of vehicles and / or pedestrians at locations such as traffic intersections. In such an embodiment, the system 10 can be arranged to detect events such as near-crash based on the calculated proximity or proximity of the identified vehicles and / or pedestrians. The system 10 can also identify other dangerous or troublesome situations by identifying the presence or location of pedestrians, vehicles or other objects. For example, the system 10 can determine whether the vehicle is not stopped at a stop sign, whether the intersection will benefit from increased traffic control equipment (e.g., traffic lights, stop signs), whether pedestrians use non-designated areas to cross the road, and so on. If such a dangerous or troublesome situation is detected, the system 10 can be arranged to send a signal (e.g., a message or warning) to a designated computer, system, address (e.g., email address) or personnel responsible for traffic safety via the network 15.

[0037] In another embodiment, the system 10 is a self-driving or traffic management system, wherein each node 12 takes the form of a car or an accessory for a car (e.g., a dash cam, a backup camera, etc.). In one embodiment, the system 10 is arranged for use in buildings such as banks, warehouses, government buildings, etc., and is arranged to identify objects or individuals that indicate an intrusion or security threat. In one embodiment, the system 10 is used to monitor wildlife in a particular environmental habitat. Those skilled in the art will readily recognize a variety of other uses for the system 10.

[0038] The centralized server 14 is any combination of software and / or hardware that enables the centralized server 14 to provide functionality to the network 15, the edge node 12, or other devices or nodes on the network 15. In one embodiment, the centralized server 14 includes a processor 26, a memory 28, and a network interface 30, which are generally similar to the processor 16, the memory 18, and the network interface 20, respectively. However, the computing resources of the edge node 12 are relatively limited compared to the computing resources of the centralized server 14. That is, the speed of the processor 16, the data storage size of the memory 18, etc. are significantly smaller than the corresponding components of the centralized server 14. In one embodiment, the centralized server 14 is or includes a group of computers or computing resources that are "cloud-based" or can be obtained from a cloud-based computing solution. Therefore, the centralized server 14 can perform complex computing tasks that cannot be performed by the edge node 12.

[0039] Due to the relatively limited computing power of the edge nodes 12, and as discussed in more detail below, the ANNs 24 in the disclosed embodiments are pruned versions of the parent ANNs 32 included in the centralized server 14. To this end, the centralized server 14 may include a pruning application 34 that is deployed to delegate pruned ANNs to each edge node 12, and the ANNs 24 may be referred to as "pruned". The parent ANN 32 may take any of the forms mentioned above, such as a CNN, or an ANN based on another deep learning model that is capable of detecting and classifying objects in an image.

[0040] It is recognized in the current embodiment that each edge node 12 is located at a specific physical location, and therefore the camera 22 only captures images corresponding to the geographic area associated with a specific one of the edge nodes. Figure 2 One example scenario that may benefit from the presently disclosed embodiments is illustrated in which the edge nodes 12 are each light emitters (eg, street lamps, lamp posts, traffic lights, etc.) in a networked lighting system. Figure 2 Each of the edge nodes 12 is provided with an alphabetical identifier (ie, 'a', 'b', 'c', 'd') appended to the number 12 to facilitate discussion regarding the different edge nodes 12 . Figure 2 The scenario includes three of the edge nodes 12 (edge ​​nodes 12a, 12b, and 12c) arranged to observe a road intersection, and one of the edge nodes 12 (edge ​​node 12d) arranged to observe a sidewalk through a park.

[0041] Each camera 22 corresponding to an edge node 12 has a certain field of view, typically in the form of a cone extending from the camera. Figure 235 for the camera corresponding to the edge node 12a. Thus, the field of view 35 defines the geographic area corresponding to the edge node 12a. Each of the other edge nodes 12 will include a similar field of view that defines the geographic location corresponding to each respective one of the edge nodes 12. Where necessary, the image captured by the camera of each edge node 12 corresponds to the geographic area defined by the field of view corresponding to the camera.

[0042] It is expected that different edge nodes at different locations will encounter different objects, and no edge node will encounter all of the objects that a comprehensive ANN, such as parent ANN 32, has been trained on (e.g., current ANNs may be trained on over a thousand different object categories). For example, edge node 12d (which overlooks a sidewalk in a park) may see objects that are typically found in a park, such as a dog, a person, a cyclist, or a ball. In contrast, edge node 12a, which overlooks a traffic intersection, may see objects such as cars, trucks, motorcycles, etc. Figure 2 It is unlikely that any edge node 12a-12d in the scene will encounter objects corresponding to even a portion of the entire classification for which the current ANN has been trained, such as various exotic animals (e.g., stingrays, polar bears, etc.), food (e.g., bananas, pineapples, etc.), household objects (e.g., toilet paper, televisions, etc.). Since edge nodes may be provided in any location, the objects encountered will vary greatly from edge node to edge node. An edge node with a camera positioned to overlook an Arctic landscape may never see a car (just as edge nodes 12a-12d will never see a polar bear), but can be used to help identify and track polar bear populations (just as edge nodes 12a-12d can be used to track traffic flow).

[0043] Figure 3 A method 40 is provided in which the ANN 24 of each edge node 12 can be customized for each specific edge node 12 based on the images actually captured at the camera 22 of the corresponding edge node. The customization results in only those filters corresponding to the objects actually encountered by each edge node 12 being included in the ANN 24 of the edge node. In this way, the method 40 can be considered a "bottom-up" pruning method because the selected filters are determined based on the objects recognized in the actual images captured by the camera at each edge node, and the pruned ANN 24 is constructed on a filter-by-filter basis using only these filters. This results in each ANN 24 being streamlined in a customizable manner for its corresponding edge node.

[0044] The method 40 begins at step 42, in which, for each relevant edge node 12, a plurality of images (which are referred to herein as baseline images) are captured by the camera 22 of that edge node 12. As should be appreciated in view of the above, an important purpose of the baseline images is to create a theme of objects actually encountered within the geographic proximity of each edge node 12. Therefore, the baseline images should be collected over a sufficient period of time to collect a representative sample of different objects in the geographic area corresponding to each edge node 12. In one embodiment, this is done over a period of at least several days.

[0045] At step 44, the reference image is sent to the centralized server 14. The reference image is then analyzed by the parent ANN 32 of the centralized server 14 at step 46. A pruned copy of the parent ANN 32 (i.e., one of the ANNs 24) is then created at step 48 based on the identified objects / filters identified in the analysis of step 46. The pruned ANN 24 is then sent to the edge node that originally captured the reference image at step 50. The edge node 12 then captures the image in real time at step 52, and uses the pruned ANN at step 54 to identify objects in the image. By "real time" is meant that the image is captured by the edge node and then analyzed by the edge node, as opposed to being captured, sent to another device (such as the centralized server 14) for processing, and then the results are transmitted back to the edge node. The analysis of the image can be performed immediately after the capture of the image or simultaneously with the capture of the image, or after an appropriate delay (e.g., at a scheduled time of day or when the edge node 12 detects that it has entered an idle state and / or has idle processing power available for use).

[0046] Steps 42-50 may be referred to or understood as a pruning routine 56, while steps 52-54 may be referred to or understood as an operating routine 58. In one embodiment, the pruning routine 56 is performed only once upon initial setup of the edge node 12, while the operating routine 58 is performed repeatedly and indefinitely thereafter. In one embodiment, the system 10 is configured to run the pruning routine 56 occasionally over time (e.g., once a week, once a month, once a year, etc.) or upon user input (e.g., via a command issued from the centralized server 14) to ensure that the ANN 24 of the edge node 12 remains properly configured (e.g., to benefit from enhancements made to the parent ANN 32, to accommodate changes in the geographic area surrounding each edge node 12, etc.).

[0047] Figure 4 FIG. 1 is a flowchart further describing the operation of the system 10 and the pruning routine 56 of the method 40 according to one embodiment. In accordance with step 44 of the method 40 discussed above, Figure 4A reference image, designated with reference numeral 60 in FIG. 1 , is first provided to a parent ANN 32, for example, from one of the edge nodes 12. The parent ANN 32 then analyzes the reference image, for example, according to step 46. In addition to storing the deductions made during the analysis, the parent ANN 32 may create or pass information to a frequency map or histogram, designated with reference numeral 62. The frequency map 62 tallies or counts the number of times each different object class was identified during the analysis performed by the parent ANN 32. The frequency map 62 may be used to determine a subset of objects, designated with reference numeral 64, that were identified in the reference image (and therefore actually seen by the edge node that captured the reference image).

[0048] The parent ANN 32 and the subset 64 are then transmitted to the pruning application 34 at step 66. The pruning application 34 may be a software application specifically configured to perform step 48, such as being installed in the memory 28 and executed by the processor 26. In addition, training images 68 may be provided to the pruning application at step 69. The training images 68 include known or verified examples of each object in the subset 64. For example, if "car" is one of the objects in the subset 64, the training images 68 will contain known images of cars. In one embodiment, the training images 68 have been manually reviewed by a human and marked as belonging to a specific object category.

[0049] The pruning application 34 performs step 48 by running the training image 68 through the parent ANN at step 70. The pruning application 34 identifies filters corresponding to objects in the training image 68 at step 72. In one embodiment, only filters that respond in a sufficiently strong manner, such as with a maximum response, are selected. The identified filters are compiled into a list at step 74. At step 76, the process is repeated for each object in the subset 64. Once the reference images corresponding to all objects are processed at step 76 and their corresponding filters are identified, step 48 is completed and a pruned ANN 24 is created by constructing the pruned ANN 24 using only those filters identified in step 74. The pruned ANN 24 is then deployed at step 78 by sending the pruned ANN 24 to a corresponding one of the edge nodes 12 that captured the reference image 60. This process is repeated individually for each edge node 12 to customize the pruned ANN 24 for each edge node 12.

[0050] The ANN 24—which is a customized pruned ANN built solely from the identified filters of the parent ANN 32—enables the relatively resource-limited edge node 12 to use the ANN 24 to infer objects identified in images captured by the camera 22 of the edge node 12 in a timely and feasible manner. Figure 2In the case of edge nodes 12a, 12b, and 12c, it should be appreciated that the ANNs of edge nodes 12a, 12b, and 12c should be similar if not identical because each of these edge nodes observes substantially the same geographic area. However, edge node 12d does not observe a traffic intersection, and therefore its pruned ANN may not be similar to the pruned ANNs of edge nodes 12a, 12b, and 12c. Advantageously, the ANN of each edge node 12 is thus customized so that each edge node 12 can achieve maximum computational speed without sacrificing the accuracy of the ANN 24.

[0051] Also return reference Figure 2 , additional embodiments can be realized about edge nodes 12a, 12b and 12c. That is, these nodes all observe similar areas, namely the illustrated traffic intersection. The object subsets (e.g., subset 64) corresponding to each of edge nodes 12a, 12b and 12c are expected to be similar (because all observe the same area) and can therefore be further divided so that each of edge nodes 12a, 12b and 12c is responsible for only a portion of the object subset. For example, if the object subsets corresponding to nodes 12a, 12b and 12c include "cars", "pedestrians" and "motorcycles", the ANN of edge node 12a can be constructed to include only filters corresponding to the "car" portion, while the ANN of edge node 12b includes only filters for "pedestrians", and the ANN of edge node 12c should be constructed to include only filters corresponding to "motorcycles". Since edge node 12 is part of a communication network such as network 15, edge nodes 12a, 12b and 12c can communicate the results of their respective analyses to each other.

[0052] Although several inventive embodiments have been described and illustrated herein, it will be easy for those skilled in the art to envision a variety of other devices and / or structures for performing functions and / or obtaining the results and / or one or more advantages described herein, and each of such variations and / or modifications is considered to be within the scope of the inventive embodiments described herein. More generally, it will be easy for those skilled in the art to appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary, and actual parameters, dimensions, materials, or / or configurations will depend on the specific one or more applications to which the inventive technology is used. Those skilled in the art will recognize or be able to confirm many equivalents for the specific inventive embodiments described herein by using no more than routine experiments. Therefore, it should be understood that the above-mentioned embodiments are presented only as examples, and within the scope of the appended claims and their equivalents, the inventive embodiments can be practiced differently from the specific description and protection claimed. The inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the inventive scope of the present disclosure if such features, systems, articles, materials, kits, and / or methods are not inconsistent with each other.

[0053] All definitions, as defined and used herein, should be considered controlling over dictionary definitions, definitions in documents incorporated herein by reference, and / or ordinary meanings of the defined terms.

[0054] The indefinite article "a" or "an" as used herein in the specification and claims should be construed to mean "at least one", unless explicitly stated to the contrary.

[0055] The phrase "and / or", as used herein in the specification and claims, should be understood to refer to "either or both" of the elements so combined, i.e., elements that exist together in some cases and separately in other cases. Multiple elements listed by "and / or" should be interpreted in the same way, i.e., "one or more" of the elements so combined. Other elements other than the elements specifically identified by the "and / or" clause may optionally be present, whether related or unrelated to those elements specifically identified. Therefore, as a non-limiting example, a reference to "A and / or B", when used in conjunction with an open language such as "including", refers to only A (optionally including elements other than B) in one embodiment; to only B (optionally including elements other than A) in another embodiment; and to A and B (optionally including other elements) in yet another embodiment, and so on.

[0056] As used herein in the specification and claims, "or" should be understood to have the same meaning as "and / or" defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive, that is, including at least one of many elements or lists of elements, but may also include more than one element, and optionally, including items that are not listed in addition. Explicitly indicating the opposite terms, such as "only one" or "just one", or "consisting of..." when used in the claims will refer to just one element included in many elements or lists of elements. Generally, the term "or" as used herein, when in front of an exclusive term such as "any one", "one", "only one" or "just one", will only be interpreted as indicating an exclusive alternative (that is, "one or the other, but not both"). When used in the claims, "consisting essentially of..." will have its ordinary meaning as used in the field of patent law.

[0057] As used herein in the specification and claims, the phrase "at least one" when referring to one or more elements should be understood to mean at least one element selected from any one or more elements in a list of elements, but not necessarily including at least one element of each element specifically listed in the list of elements, and not excluding any combination of elements in the list of elements. This definition also allows that elements other than the elements specifically identified in the list of elements to which the phrase "at least one" refers may optionally be present, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B", or, equivalently, "at least one of A and / or B") may be: in one embodiment, referring to at least one (optionally including more than one) A, and no B is present (and optionally including elements other than B); in another embodiment, referring to at least one (optionally including more than one) B, and no A is present (and optionally including elements other than A); in yet another embodiment, referring to at least one (optionally including more than one) A, and at least one (optionally including more than one) B (and optionally including other elements), and so on.

[0058] It should also be understood that in any method claimed herein that includes more than one step or action, the order of the steps or actions of the method is not necessarily limited to the order in which the steps or actions of the method are recited unless explicitly indicated to the contrary.

[0059] In the claims and the above description, all transitional phrases such as "including," "comprising," "carrying," "having," "containing," "involving," "having," "consisting of," etc. are to be understood as open-ended, meaning including but not limited to. As set forth in Section 2111.03 of the U.S. Patent Office's Manual of Patent Examining Procedures, only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively.

Claims

1. A method for object recognition, comprising the steps of: capturing a plurality of reference images (60) using a camera (22) associated with an edge node (12) on a communication network (15); Receiving the reference image using a centralized server (14) on the communication network; Analyzing the reference image using a parent neural network (32) of the centralized server to determine a subset of objects (64) in the reference image that are identified by the parent neural network, wherein the parent neural network comprises a convolutional neural network (CNN); selecting one or more filters in the parent neural network that are responsive to the subset of objects, the one or more filters configured to activate in response to patterns in the reference image; Creating a pruned neural network (24) based on the one or more filters produced by the selecting step, and wherein the pruned neural network comprises a convolutional neural network (CNN); deploying the pruned neural network to the edge node; Capturing real-time images using a camera of the edge node; and Objects in the real-time image are recognized using the pruned neural network. 2 . The method according to claim 1 , wherein the method is performed for each of a plurality of edge nodes each connected to the communication network.

3. The method of claim 2, wherein the cameras of the first edge node (12a) and the second edge node (12b) are configured to observe a common geographic area, the method further comprising dividing the object subset into a first portion and a second portion, such that the pruned neural network of the first edge node is created based on those filters corresponding to the first portion, and the pruned neural network of the second edge node is created based on those filters corresponding to the second portion.

4. The method of claim 2, wherein each of the edge nodes comprises a light emitter (12a-12d) of a networked lighting system.

5. The method of claim 4, wherein the networked lighting system comprises one or more lamp posts, street lights or traffic lights, and wherein the method is used to enhance pedestrian safety.

6. The method of claim 1, wherein the selecting step comprises analyzing a plurality of known training images using the parent neural network (68).

7. The method of claim 1, wherein selecting one or more filters comprises selecting a filter that exhibits a maximum response to a known training image. The method according to claim 1 , wherein the plurality of reference images are formed by one or more videos.

9. The method of claim 1, wherein the analyzing comprises creating a frequency map (62) by counting the number of times each object category is recognized by the parent neural network, and determining the object subset based on the frequency map.

10. An object recognition system (10) for a communication network (15) having a plurality of edge nodes (12), comprising: A centralized server (14) for the communication network, the centralized server having: a parent neural network (32) configured to analyze the reference image received by the centralized server from each of the edge nodes to determine a subset of objects (64) in the reference image corresponding to each of the edge nodes, wherein the parent neural network comprises a convolutional neural network (CNN); and a pruning application (34) configured to select one or more filters in the parent neural network, the one or more filters configured to activate in response to patterns in the reference image and to be responsive to each subset of objects, and wherein the pruned neural network comprises a convolutional neural network (CNN); and the pruning application is further configured to create, for each of the edge nodes, a pruned neural network (24) based on the one or more filters corresponding to each particular edge node. The system of claim 10 , wherein the edge node comprises one or more light emitters.

12. The system of claim 11, wherein the one or more light emitters include one or more lamp posts, street lights, or traffic lights.

13. The system of claim 12, wherein the system is implemented to enhance pedestrian safety.

14. The system of claim 10, wherein the centralized server comprises a memory and a processor, and the parent neural network and the pruning application are installed in the memory and executed by the processor.

Images