Edge device for categorizing and selecting field image data for transmission to a central server

Abstract

An edge device acquires image data including camera images or scans of target objects, such as grains harvested by a combine, and applies a deep learning detection model to detect objects and characteristics associated with the objects in the images so that the images can be categorized into categories representing embodied characteristics associated with the target objects in the captured images. The edge device compares the embodied characteristics associated with the images to the selection criteria to determine which images are transmitted to a central server for subsequent training of the deep learning detection model on the server. An out-of-distribution detection model on the edge device can also distinguish between in- distribution images and out-of-distribution images among the captured images so that out-of- distribution images are also transmitted to the server for model training.

Description

[0001] EDGE DEVICE FOR CATEGORIZING AND SELECTING FIELD IMAGE DATA FOR TRANSMISSION TO A CENTRAL SERVER

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to an edge device and method of operation thereof in which the edge device includes a deep learning detection model for processing and selecting image data of target objects to be communicated to a central server for subsequent training of the deep learning detection model on the server.

[0004] BACKGROUND

[0005] Data plays a vital role in Machine Learning and Deep Learning applications. Self curated datasets are very helpful to create the models capable of making decent predictions; however, the actual field data is very important when generalizing the models and making them useful in industry-based applications. In some instances, the actual field data being generated is too large in quantity to efficiently process and / or transmit over limited bandwidth to the deep learning applications that create the models. In other instances, bandwidth may not be limited, but if too much data is collected at the server level, scripts must be created to run through all of the data in the database such that unnecessary server costs are incurred, particularly when a large portion of the data may be redundant and not useful for training further models.

[0006] Furthermore, (i) the actual field data may not be representative of typical conditions that result in poor training of the models, (ii) the actual field data may represent particular conditions that would be desirable for training of the models, or (iii) the data from historical data sets may be based on previous year crops and thus may no longer be relevant to what is appearing in the current season. In such instances, it would be desirable to prioritize some field data over other field data for use in training the models.

[0007] SUMMARY OF THE INVENTION

[0008] According to one aspect of the present invention there is provided an edge device for communication over a communication network with a central server located remotely from the edge device, the edge device comprising: a controller having a processor and a memory storing selection criteria and programming instructions thereon; a data acquisition device arranged to supply captured image data to the controller in which the captured image data comprises captured images of target objects; a deep learning detection model residing on the controller and being operable to categorize the captured images into categories which represent one or more embodied characteristics associated with the target objects in the captured images; a communication device arranged to transmit selected images among the captured images from the controller over the communication network to the server; wherein the controller is operable to execute the programming instructions so as to be arranged to (i) for each captured image, use the deep learning detection model to identify which of the one or more embodied characteristics are associated therewith, (ii) determine the selected images to be transmitted among the captured images by comparing the one or more embodied characteristics associated with the captured images to the selection criteria, and (iii) direct the selected images to the communication device for transmission to the server together with the one or more embodied characteristics associated therewith.

[0009] The method and edge device described herein serves to notify the field units of things of interest to quickly have a narrowed in data set that is flagged when it gets to the server so that we can use edge-deployed hardware to keep server costs down. By classifying the field data, that is the captured images, on the edge device using the deep learning model and then making selections using criteria before transmitting the data to a central server where the deep learning model is further trained, the algorithm or model can be more efficiently trained in a more purposeful manner with regard to typical traits that the algorithm is expected to encounter. For example, when the harvest season starts, we may find that specific detection models are underperforming. This is especially possible if we use the data from our historical data sets to train new models that will be based on previous year crops and potentially not that relevant to what is appearing in the current season. If the field devices are instructed to select specific images according to specified criteria, the system can quickly train and deploy models that are relevant to the current growing season and or regional issues. Overall, current data is more useful than old data when training models to identify things in the same year.

[0010] The processing of the image data and use of selection criteria at the edge device also allows for more efficient use of bandwidth in particular instances where bandwidth is limited.

[0011] In a preferred embodiment, the target objects comprise grains in a captured image of a grain sample.

[0012] The data acquisition device may be an optical camera in which the captured images are optical images, or the data acquisition device is an infrared device in which the captured images are spectral scans.

[0013] The edge device is preferably arranged to be supported on agricultural equipment and the target objects in the captured images corresponding to grains captured in real time as the grains are conveyed by the agricultural equipment.

[0014] The controller is preferably arranged to receive instructions from the server and change the selection criteria stored on the controller in response to the received instructions.

[0015] The selection criteria may correspond to selection of images desirable for training of the deep learning model by the server.

[0016] The controller may be operable to execute the programming instructions so as to be further arranged to queue the selected images for subsequent transmission to the server at a later time. When the selection criteria include a bandwidth limitation, the controller may be operable to execute the programming instructions so as to be further arranged to determine a quantity of the selected images among the captured images based upon the bandwidth limitation. When a first portion of available bandwidth is dedicated for non-image data communications between the controller and the server, the quantity of the selected images among the captured images may be based on a remaining portion of the available bandwidth. The controller may be further operable to execute the programming instructions so as to be further arranged to (i) determine the selected images among the captured images so as to satisfy the selection criteria in real time as the images are captured, and (ii) queue the selected images for subsequent transmission to the server when bandwidth is available. The controller may be arranged to determine the selected images among the captured images to meet the bandwidth limitation by random selection.

[0017] In some instances, the selection criteria correspond to a presence of an identified one of the one or more embodied characteristics being associated with one or more of the target objects in the captured image.

[0018] The selection criteria may also correspond to a quantity of the target objects in the captured image that possess an identified one of the one or more embodied characteristics that falls outside of a normal threshold range relating to the occurrence of said one or more embodied characteristics.

[0019] The selection criteria may also correspond to an identified one of the embodied characteristics that is representative of the captured image as a whole being outside of a normal threshold range.

[0020] When the edge device is used together with the server, the server is preferably arranged to communicate modified criteria to the controller to replace the selection criteria stored on the criteria in response to a user instruction submitted to the server. When the selection criteria further include at least one first criterium to be met and at least one second criterium to be met, the controller may be arranged to initially select captured images to satisfy the first criterium, and upon satisfying the first criterium the control is subsequently arranged to select captured images to satisfy the second criterium.

[0021] Preferably the edge device further comprises an out-of-distribution detection model residing on the controller and arranged to distinguish between in-distribution images and out-of-distribution images among the captured images, and wherein the controller is further arranged to (i) use the out-of-distribution detection model to identify the out-of-distribution images among the captured images, and (ii) direct the out-of-distribution images to the communication device for transmission to the server. The controller is preferably arranged to direct intermediates outputs from the deep learning detection model as inputs to the out-of- distribution detection model, in which the intermediate outputs comprise embeddings from the deep learning detection model in the form of data vectors extracted from an intermediate layer of the deep learning detection model.

[0022] According to a second aspect of the present invention there is provided a method of classifying and selecting field image data for transmission to a central server, the method comprising: providing an edge device located remotely from the central server in which the edge device includes (i) a controller having a processor and a memory storing selection criteria and programming instructions thereon, (ii) a data acquisition device, (iii) a communication device, and (iv) a deep learning model residing on the controller; using the data acquisition device to supply captured image data to the controller in which the captured image data comprises captured images of target objects to be classified; operating the controller to (i) categorize the captured images into categories which represent one or more embodied characteristics associated with the target objects in the captured images, (ii) determine selected images among the captured images by comparing the embodied characteristics associated with the captured images to the selection criteria, and (iii) using the communication device, transmit the selected images among the captured images from the controller over the communication network to the server.

[0023] The method may further include changing the selection criteria stored on the controller in response to instructions from the server based on features identified in the selected images.

[0024] When the edge device further comprises an out-of-distribution detection model residing on the controller and arranged to distinguish between in-distribution images and out- of-distribution images among the captured images, the method may further include (i) operating the controller to (a) use the out-of-distribution detection model to identify the out-of-distribution images among the captured images, and (b) using the communication device, transmit the out- of-distribution images to the server, and (ii) changing the selection criteria stored on the controller in response to instructions from the server based on features identified in the out-of- distribution images.

[0025] BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Some embodiments of the invention will now be described in conjunction with the accompanying drawings in which:

[0027] Figure 1 is a schematic representation of a system including an edge device employing a deep learning model for managing classification, and selection of field data for transmission to a central server that further trains the deep learning model;

[0028] Figure 2 is a schematic representation of the edge device for use within the system according to Figure 1 ;

[0029] Figure 3 is a schematic representation of the method of operation of the system according to Figure 1 ; Figure 4 is a schematic representation of the operation of the edge device according to Figure 1 in which the out-of-distribution model is shown operating in parallel with the deep learning detection model using the same initial image data collected by a camera or scanner; and

[0030] Figure 5 is a schematic representation of the out-of-distribution model of Figure 4, shown using embeddings output from the deep learning detection model as an input for the out-of-distribution model.

[0031] In the drawings like characters of reference indicate corresponding parts in the different figures.

[0032] DETAILED DESCRIPTION

[0033] Referring to the accompanying figures there is illustrated a system 10 including an edge device 12 that uses a deep learning model for managing classification and selection of field data for subsequent transmission to a central server 18 that is capable of further training the deep learning model. More particularly, the edge device 12 of the system 10 uses the deep learning model to classify the data according to classes that represent embodied characteristics associated with the image data, and then further select images among the classified image data by comparing the associated embodied characteristics to selection criteria. The selection corresponds to data that is better suited for transmission to the central server for optimal training of the deep learning algorithm or model at the server.

[0034] As shown in figure 1, the system 10 may include a variety of different edge devices 12 such as (i) a first embodiment of the edge device 12A arranged to be mounted on mobile field equipment such as a combine harvester 14, an auger 15, or other grain handling equipment, or (ii) a second embodiment of the edge device 12B arranged to be mounted in a fixed environment for capturing sample data within a laboratory, shop or other industrial facility by mounting the edge device in association with a benchtop measurement device 11 or on a grain handling auger 13 or leg system that moves grain. In each instance, the edge device 12 communicates over a communication network to a server 18, such as a cloud-based server arranged for communication over the Internet or another similar communications network with the edge devices located remotely from the server.

[0035] In the instance of edge devices 12A mounted on mobile grain handling equipment or edge devices 12B mounted on stationary equipment, the edge device takes images and / or infrared scans of the grain and then runs them through the models that have been loaded onto the device from the server. The device presents results as soon as they are ready for the local interface and send the results back to the server. In instances of limited bandwidth according to some instances of a mobile edge device 12 A, the images and scans can be stored for a predetermined amount of time and the decision of which data to be communicated back to the server may be dependent upon accommodating the available bandwidth. The server can request to get associated images or statements sent to it through one of the mechanisms outlined based on the information from the results.

[0036] The edge device 12A can be mounted on a combine harvester by being either located on the clean grain elevator, or the bubble up auger, or somewhere else where grain is handled for imaging. Other mobile equipment including augers and the like, the device 12A can be mounted at any suitable location to image and scan the grain being conveyed with a camera or infrared sensors and the like. Suitable electronics are also provided to run the models and communicate to the server via a suitable communications network including cellular, Wi-Fi or through an interface device 20 with which it communicates.

[0037] The edge devices 12B are similarly mounted so that cameras or infrared sensors and the like are also able to image and scan the grain. Again, suitable electronics are also provided to run the models and communicate to the server via a suitable communications network including cellular, Wi-Fi or through an interface device 20 with which it communicates. The cloud-based server 18 is also arranged for communication with a plurality of remote user devices 20 such as user computers including smart phones, desktop computers, laptops, tablets and the like. The remote user devices 20 comprise computers having programming instructions stored on a memory, a processor for executing the programming instructions, and communication capability for communicating with the server 18 over the communications network 16. The remote user devices enable authorized users to access the data on the server for viewing data and for submitting requests or instructions from the user to the server to generate suitable instructions to execute the various functions described herein. These remote devices 20 or interfaces include mobile and web interfaces both customer-facing and for internal usage or third-party interaction.

[0038] The remote user devices 20 can also include mobile or personal computer devices, for example tablets and smartphones that can communicate directly with the edge devices 12A or 12B. When communicating directly with an edge device, the user computer device 20 may be a tablet, cell phone or interface of some kind in the cab of the combine or at the operating controls of the equipment for viewing live results connected via hardware or wireless to the hardware.

[0039] The cloud-based server 18 is arranged to store results from the edge devices as well as images are all communicated back to the server in real time on an ongoing basis including any additional information associated with the images as described in the following. The cloudbased server deploys data requests for image data from the edge devices based on customizable data such as content of the images, or quantity of contents and the like. The server 18 also makes data available to remote interface devices 20 so that authorized users logging in through the remote interfaces can view, annotate and direct training of models for example. The server can initiate individual requests for images or send out instructions that remain on the devices until the criteria is met and then send the images or scans. Turning now to figure 2, each edge device 12A / 12B is a computer device having a computer memory 22 for storing data and programming instructions thereon in which the programming instructions are executable by a processor of the edge device to execute the various functions of the edge device as described herein. The edge device further includes a camera interface 24 arranged for communication with a separate camera 26 for capturing optical or infrared images of the target objects in real time. The camera may be a conventional peripheral device arranged for connection to the edge device 12 using a USB connection and the like. In the instance of a combine harvester, the camera 26 is mounted on the harvester for capturing a time series set of captured images in which the target objects within the images comprise individual grains within a flow of grain at a prescribed location within the operating system of the harvester such as the grain flowing through the clean grain elevator as an example.

[0040] In the instance of a fixed mounted edge device 12, an operator may load individual grain samples into a viewing area of the camera to capture images of the grain samples in which the target objects within the captured images again comprise individual kernels of grain.

[0041] The edge device 12 also includes a communication device or module 28 arranged to communicate with the communications network 16 for exchanging information with the cloud-based server 18 by sending data over the network to the server and by receiving instructions from the server.

[0042] The edge device 12 further includes a hardware control module 30 which provides an interface to communicate with various external control hardware 32 such as control systems of the combine harvester such that the operation of the edge device can respond to operating conditions of the mobile field equipment 14 upon which it is installed. For example, the operating condition of the edge device may be dependent upon the operating condition of the combine harvester 14 upon which the edge device is mounted for operation. A control interface 34 of the edge device is arranged for communicating information between the edge device and an external device 36 such as a user computer including a tablet computer and the like. The user of the device can thus use their personal external computer device 36 to interact with the edge device for changing the programming or configuration of the device for example.

[0043] A deep learning module 38 of the edge device includes the deep learning algorithm or model thereon so as to be arranged for processing the captured images acquired from the camera to detect objects in the images and classify the detected objects into one or more classes that each represent an embodied characteristic of the images, or of individual target objects (for example kernels of grains) within the images. In either instance, the images are effectively categorized into categories which represent one or more embodied characteristics associated with the target objects in the captured images. This enables the edge device to make various decisions with regard to the content of the images including identifying one or more image characteristics associated with the captured images. The embodied image characteristics may relate to (i) characteristics of the individual target objects or kernels of grain within the captured images or (ii) characteristics with regard to the entire image as a whole. For example, characteristics of individual target objects include the condition of individual kernels of grain such as the presence of damage or disease, the quantity of kernels of grain possessing one or more conditions, or a general rating relating to a characteristic or quality of the individual kernels of grain. Characteristics relating to the sample as a whole may relate to an average rating representing an average condition or quality rating of the individual grains or various measured ratios such as a ratio of grains having an identified characteristic relative to grains without the identified characteristic within the sample image as a whole.

[0044] The edge device 12 as described herein further includes an out-of-distribution detection system (ODDS) module 50 that receives an input from the deep learning module 38 based on the same captured image data. The results of the ODDS module 50 are communicated by the communications device 28 to the server together with other images that meet the selection criteria stored on the edge device. The ODDS module 50 is a machine learning based learning algorithm or model that is trained to distinguish between in-distribution data, which the model has seen during training, and out-of-distribution data, which it has not seen. The main detection model 38 is used to generate embeddings 52 (intermediate outputs) for all data to capture key features for each sample. The embeddings 52 are data vectors that are extracted from inside the detection model 38 as shown in Figure 5. This can be accomplished by chopping the model 38 off a few layers before the main output of the detection model 38. This chopped off model will still accept image data as input, but the output will be vector or group of vectors referred to herein as embeddings 52. The ODDS module 50 is trained using the embeddings 52 to detect the outliers. Once trained, this system helps identify the out of distribution data points.

[0045] Subsequent to extraction of the embeddings 52 from the detection model 38 to the ODDS model 50, the detection model 38 continues to process the captured images to identify the target objects within the images and detect characteristics associated with the images including detection of diseased or damaged kernels for example. A grading system 54 of the detection model 38 categorizes the captured images according to categories which represent one or more embodied characteristics associated with the target objects in the captured images and then compares the one or more embodied characteristics associated with the captured images to the selection criteria to determine which images are selected as results to be transmitted to the server as selected images flagged with the one or more embodied characteristics associated therewith. Any images that are flagged as out-of-distribution images by the ODDS model 50 are also flagged as being out-of-distribution and those images are added to the selected images to be transmitted to the server (if not already selected for transmission).

[0046] Each edge device 12 as shown in figure 2 may comprise a singular computer device locating all of the various components 22, 24, 26, 28, 30, 32, 34 and 38 thereon, or more preferably the edge device 12 comprises multiple computer devices which are separate from one another but in communication with one another to collectively provide the functions of the edge device as described herein.

[0047] Once the individual target objects within the captured images, or the captured images as a whole have been identified and classified by the deep learning module, the processed and classified images along with the identified characteristics associated therewith can be compared to various selection criteria to determine which captured images meet the criteria to be selected for transmission over the communications network to the server 18.

[0048] The selection criteria are stored on the edge device and can be updated by instructions received from the server over the communications network. Typically, an operator of the server or programming instructions executed by the server will identify a class of desirable images which are desired for optimally training one or more deep learning algorithms or models residing on the server. A set of criteria will then be established to target desirable images and this identified set of criteria is communicated to the edge devices of the system so that the edge devices will process the images to determine which of the captured images should be selected as desirable images for transmission to the server to optimally train the categorizing and detection algorithms or models stored on the server.

[0049] Figure 3 schematically represents the operation of the system 10 in which the classification algorithm or model 40 on the server 18 is trained on initial data sets derived from a first group of sample images; however, as the system continues to operate, additional field data in the form of captured images are processed and classified by the deep learning algorithm residing on the edge devices to determine which captured images are most desirable for improving the training of the models 40 stored on the server. These selected images are communicated to the server for training of the detection algorithms or models 40 stored on the server. As the model(s) 40 are trained on new data sets, the criteria as to which additional data sets are most desirable to further improve the training are continuously updated and the updated criteria are communicated back to the edge devices 12 to replace the existing selection criteria that aids in selecting the next most desirable data sets for training of the model. The updated models 44 for classifying the captured images are also communicated back to the edge devices for updating the deep learning module 38 of the edge device to improve the classification of the images and identification of image characteristics within the captured images as performed by the edge device.

[0050] Figure 4 schematically represents the operation of ODDS model 50 by the controller of the edge device 12 in parallel with the detection model 38. The ODDS model 50 uses data from an intermediate layer of the detection module 38, that is feature embeddings 52 from the already in-use detection models 38 that are fed into the ODDS model 50. The ODDS model 50 analyzes the data and determines whether the incoming data is in-distribution or out- of-distribution. The ODDS model similarly resides on the controller of the edge device 12 and is arranged to distinguish between in-distribution images and out-of-distribution images among the captured images. The controller uses the out-of-distribution detection model to identify the out-of-distribution images among the captured images, and then direct the out-of-distribution images to the communication device for transmission to the server along with associated data to identify the transmitted images as being flagged as an out-of-distribution image. More particularly, the model 50 can identify if something is non-detectable or out of place in an image, so that the corresponding image can be flagged as an image of interest and prioritize it through the method of optimizing bandwidth or through the desire to have current data to train with. When the images are flagged with ODDS, the image may be one of significant interest because it is known that these images represent features that are specifically underperforming. These flagged images are used similarly to the images selected by selection criteria by the detection model 38, that is the flagged images are used as training images to improve existing models 44 on the server which are in turn used to update the models 38 on the edge devices 12. The images flagged by the ODDS model 50 as out-of-distribution images may also provide insight into new classifications that we need to create models for or to establish new selection criteria for selecting further images with similar traits.

[0051] As described herein, the edge devices 12 may take the form of two types of hardware units available to the customers and future applications will become available, including a benchtop unit at a fixed location within an industrial facility or a field unit mounted on a combine, auger, conveyor, or other grain handling equipment.

[0052] The benchtop unit is designed to work in lab environments where internet connectivity is available. All the samples scanned through the benchtop units will be available for model improvements via cloud storage because of seamless internet connectivity in lab environments. The On-combine unit, however, is supposed to be mounted and operated on combine during harvest in settings where internet bandwidth could be a resource to be managed. Since, edge devices are used to perform inferencing in the on-combine unit, the images are stored in storage interfaced with edge devices. The need to access those images along with limited internet bandwidth makes it necessary to develop a process that efficiently transfers the images of concern in limited bandwidth governed by one or more of the following described methods.

[0053] Customers may be given the option to choose the subscription for various bandwidth plans per period (period can be a day, week, month, or any set time). If the end user has selected or is defaulted to a 1 GB per month plan, we would allow for a certain percentage (ie:20%) to be carved out for model improvement overhead each month. This percentage will be determined by several factors like image acquisition frequency, size of the image, storage capacity at the edge, and bandwidth utilization of other processes. With these parameters assessed, the system can (i) either send 1 image every X minutes up to the cloud, balancing the bandwidth with other processes along the way, or (ii) store the image on edge storage and send them up the cloud in shut down sequence.

[0054] Alternatively, if prioritizing images of interest and it is possible to allow the users accounts to overrun their subscription plan, but the users would not be charged for any data overages in this instance.

[0055] Mechanisms are deployed by the edge devices using programmable selection criteria to pick the candidate images for transferring. The mechanisms will be governed by any of the following criteria, solely or in combination.

[0056] (1) The contents of the images. For example, if an image contains kernels currently required to improve poorly performing models and can be included in training sets, it will be prioritized over other images.

[0057] (2) The quantity of contents. For example, image becomes an image of interest if a specific kernel type shows up above a defined percentage.

[0058] (3) The composition of contents. For example, if the type of kernels in the image has abnormal (anomalous) proportions, it will be marked as image of interest.

[0059] (4) Overall image. For example, if an image looks different then an average image received, it will be marked as image of interest.

[0060] (5) We could also decide to just send random images to the cloud for verification.

[0061] The process described above is to be the default process and can be overridden based on needs by issuing the commands from server in case the need arises for X number of images of specific characteristics. The server can send out a request to one, some or all the field units asking them to mark the images with specific characteristics or combination of characteristics that have been identified. The server would set a number of images for each specific characteristic or combination of characteristics it is looking for and then notify the same units when its requirement is met. More than one command like this can be cued up at the same time so field units could be watching for multiple different characteristics at the same time. The server can also override overhead restriction and just send images as server requires with server commands.

[0062] Although the benchtop does not have the same constraints as the on-combine version from bandwidth consumption there is a concern on sheer volume of images to sort through. The same mechanism to identify the images of interest can be deployed to highlight the specific images to investigate. The same would be considered for future applications as well.

[0063] Since various modifications can be made in the invention as herein above described, and many apparently widely different embodiments of same made, it is intended that all matter contained in the accompanying specification shall be interpreted as illustrative only and not in a limiting sense.

Claims

CLAIMS:

1. An edge device for communication over a communication network with a central server located remotely from the edge device, the edge device comprising: a controller having a processor and a memory storing selection criteria and programming instructions thereon; a data acquisition device arranged to supply captured image data to the controller in which the captured image data comprises captured images of target objects; a deep learning detection model residing on the controller and being operable to categorize the captured images into categories which represent one or more embodied characteristics associated with the target objects in the captured images; a communication device arranged to transmit selected images among the captured images from the controller over the communication network to the server; wherein the controller is operable to execute the programming instructions so as to be arranged to (i) for each captured image, use the deep learning detection model to identify which of the one or more embodied characteristics are associated therewith, (ii) determine the selected images to be transmitted among the captured images by comparing the one or more embodied characteristics associated with the captured images to the selection criteria, and (iii) direct the selected images to the communication device for transmission to the server together with the one or more embodied characteristics associated therewith.

2. The edge device according to claim 1 wherein the target objects comprise grains in a captured image of a grain sample.

3. The edge device according to claim 1 or claim 2 wherein the data acquisition device is an optical camera and the captured images are optical images.

4. The edge device according to claim 1 or claim 2 wherein the data acquisition device is an infrared device and the captured images are spectral scans.

5. The edge device according to any one of claims 1 through 4 wherein the edge device is arranged to be supported on agricultural equipment and the target objects in the captured images corresponding to grains captured in real time as the grains are conveyed by the agricultural equipment.

6. The edge device according to any one of claims 1 through 5 wherein the controller is arranged to receive instructions from the server and change the selection criteria stored on the controller in response to the received instructions.

7. The edge device according to any one of claims 1 through 6 wherein the selection criteria correspond to selection of images desirable for training of the deep learning model by the server.

8. The edge device according to any one of claims 1 through 7 wherein the controller is operable to execute the programming instructions so as to be further arranged to queue the selected images for subsequent transmission to the server at a later time.

9. The edge device according to any one of claims 1 through 8 wherein the selection criteria include a bandwidth limitation and wherein the controller is operable to execute the programming instructions so as to be further arranged to determine a quantity of the selected images among the captured images based upon the bandwidth limitation.

10. The edge device according to any one of claims 1 through 9 wherein the selection criteria correspond to a presence of an identified one of the one or more embodied characteristics being associated with one or more of the target objects in the captured image.

11. The edge device according to any one of claims 1 through 10 the selection criteria correspond to a quantity of the target objects in the captured image that possess an identified one of the one or more embodied characteristics that falls outside of a normal threshold range relating to the occurrence of said one or more embodied characteristics.

12. The edge device according to any one of claims 1 through 11 wherein theselection criteria correspond to an identified one of the embodied characteristics that is representative of the captured image as a whole being outside of a normal threshold range.

13. The edge device according to any one of claims 1 through 12 in combination with the server, wherein the server is arranged to communicate modified criteria to the controller to replace the selection criteria stored on the criteria in response to a user instruction submitted to the server.

14. The edge device according to any one of claims 1 through 13 wherein the criteria include at least one first criterium to be met and at least one second criterium to be met, the controller being arranged to initially select captured images to satisfy the first criterium, and upon satisfying the first criterium the control is subsequently arranged to select captured images to satisfy the second criterium.

15. The edge device according to any one of claims 1 through 14 wherein the edge device further comprises an out-of-distribution detection model residing on the controller and arranged to distinguish between in-distribution images and out-of-distribution images among the captured images, and wherein the controller is further arranged to (i) use the out-of- distribution detection model to identify the out-of-distribution images among the captured images, and (ii) direct the out-of-distribution images to the communication device for transmission to the server.

16. The edge device according to claim 15 wherein the controller is arranged to direct intermediates outputs from the deep learning detection model as inputs to the out-of- distribution detection model.

17. The edge device according to claim 16 wherein the intermediate outputs comprise embeddings from the deep learning detection model in the form of data vectors extracted from an intermediate layer of the deep learning detection model.

18. A method of classifying and selecting field image data for transmissionto a central server, the method comprising: providing an edge device located remotely from the central server in which the edge device includes (i) a controller having a processor and a memory storing selection criteria and programming instructions thereon, (ii) a data acquisition device, (iii) a communication device, and (iv) a deep learning model residing on the controller; using the data acquisition device to supply captured image data to the controller in which the captured image data comprises captured images of target objects to be classified; operating the controller to (i) categorize the captured images into categories which represent one or more embodied characteristics associated with the target objects in the captured images, (ii) determine selected images among the captured images by comparing the embodied characteristics associated with the captured images to the selection criteria, and (iii) using the communication device, transmit the selected images among the captured images from the controller over the communication network to the server.

19. The method according to claim 18 further comprising changing the selection criteria stored on the controller in response to instructions from the server based on features identified in the selected images.

20. The method according to claim 18 wherein the edge device further comprises an out-of-distribution detection model residing on the controller and arranged to distinguish between in-distribution images and out-of-distribution images among the captured images, and wherein the method further comprises: operating the controller to (i) use the out-of-distribution detection model to identify the out-of-distribution images among the captured images, and (ii) using the communication device, transmit the out-of-distribution images to the server; and changing the selection criteria stored on the controller in response to instructions from the server based on features identified in the out-of-distribution images.

Images