Machine learning based auditory activity monitoring for livestock health and welfare

Abstract

Animal health and welfare may be monitored by a network edge device using a neural network trained on a server. A network edge device may include a sensor for perceiving conditions within an animal housing or ranch, and a processor for performing neural network processing to indicate an activity of interest detected by the sensor. Reports of activities of interest may be communicated to a server or other computing device for display on the dashboard.

Description

Background Art

[0001] The present invention generally relates to the auditory monitoring of livestock health and welfare, and more particularly to the auditory monitoring of livestock health and welfare using network edge devices that perform neural network processing.

[0002] Livestock have been raised and maintained for thousands of years for various purposes. The raising of livestock faces many variations, including adverse weather, predation, and many other problems. In modern animal husbandry, which may include a large number of animals in a single pasture or in single or adjacent pens, disease can be a particular concern for ranchers and farmers. In such cases, the care or removal of sick animals can be of particular importance.

[0003] Unfortunately, modern animal husbandry methods may also involve using a relatively small number of people to monitor the health and welfare of a relatively large number of animals. The presence of a relatively small number of people can increase the difficulty of detecting and identifying sick animals. Summary of the Invention

[0004] Some aspects of the present invention provide for the monitoring of the health and / or welfare of animals by a network edge device. The network edge device includes a sensor and a processor that implements a neural network for detecting a sensed activity or an activity of interest. Some embodiments include a filter for filtering the sensor output, and a further neural network for determining filter parameters. In some embodiments, the sensor is an audio sensor. In some embodiments, the sensor is a temperature sensor. In some embodiments, the sensor is an image sensor. In some embodiments, the sensor is an aerosol sensor. In some embodiments, the sensor is an infrared image sensor.

[0005] In some embodiments, the neural network is trained using a large-scale computing device (such as a computer or server), and the implementation of the neural network is downloaded to the network edge device. In some embodiments, the neural network for detecting a sensed activity is trained using data indicative of indicative sounds of livestock diseases and / or indicative sounds associated with adverse effects on animal welfare. In some embodiments, the neural network for determining filter parameters is trained using data indicative of the sounds of the livestock pen or pasture environment.

[0006] In some embodiments, an indication of a sensed activity of interest is provided by the network edge device to a remote computing device. In some embodiments, the indication of the report is provided to the remote computing device via the Internet. In some embodiments, the indication of the report is stored in a database maintained by one or more servers coupled to the Internet, and the information in the database is provided to the remote computing device. In some embodiments, the remote computing device presents information of a report indicative of poor animal health or welfare.

[0007] In some embodiments, the sensor is an audio sensor that provides a digital output. In some embodiments, the filter includes the analog-to-digital converter of the audio sensor, or circuitry associated with the analog-to-digital converter. In some embodiments, the filter is located external to the sensor. In some embodiments, the filter is an infinite impulse response (IIR) filter. In some embodiments, the filter is a finite impulse response (FIR) filter.

[0008] In some embodiments, the processor includes a frequency transformation processing circuit, circuitry for binning the output of the frequency transformation processing circuit, and a neural network circuit for detecting an auditory activity of interest using the binned output. In some embodiments, the circuitry is circuitry of a general-purpose processor configured by program instructions. In some embodiments, the processor is programmed by program instructions to provide frequency transformation processing, binning of the output of the frequency transformation processing, and neural network processing for detecting an auditory activity of interest using the binned output. In some embodiments, the frequency transformation processing circuit implements short-time Fourier transform (STFT) processing. In some embodiments, the processor further includes a neural network circuit for determining filter parameters of the filter. In some embodiments, the filter parameters are filter coefficients or filter weights.

[0009] Some aspects of the present invention detect multiple different forms of vocalization activities of multiple types of livestock through the classification ability of a neural network. Some embodiments utilize a machine learning noise filtering algorithm that is tuned using training information from a specific site or similar sites. That is, during the training process, the neural network is exposed to the specific environmental noise signals at the target deployment site so as to filter out these noise signals during the real-time inference process.

[0010] In some aspects of the present invention, initial neural network training is performed on the cloud. Examples of such services include, but are not limited to, AWS SageMaker, Azure training services, etc. In some aspects, the neural network and all accompanying firmware are remotely flashed to the edge device via an over-the-air (OTA) update system that fetches a.bin file on AWS S3 and flashes it to the devices on-site, for example via WiFi or LoRaWAN. In some embodiments, the firmware includes a neural network for classification of activities of interest, a neural network for noise filtering parameter determination, over-the-air (OTA) software details (such as update frequency,.bin file location, WiFi credentials, device-specific identifiers, etc.), STFT-specific code, and / or noise filtering-specific code. In some embodiments, both the classification and noise filtering networks are trained using TensorFlow. In some embodiments, the TensorFlow Lite library is used to perform compression of the neural network and flash it on the ESP32 device.

[0011] In some embodiments, real-time inference at the edge and communication to a dashboard for the user to display the results for determining corrective actions include one or all of the following activities: converting an audio signal to a digital signal; performing a short-time Fourier transform (STFT) on frames of the processed signal to produce a spectrogram; pooling the spectrogram frequencies into frequency bins; stacking and feeding the frequency bins into a noise filter neural network to output filter parameters (such as equalizer gains, filter weights, or coefficients); the equalizer gains are used to scale the coefficients of an IIR or FIR filter (or other filter) for reconstructing the denoised signal; performing an STFT on the denoised signal to produce another spectrogram; again pooling the spectrogram frequencies into frequency bins; feeding the frequency bins into a classification neural network to determine whether an auditory activity of interest has occurred; if the output of the neural network indicates an auditory activity of interest, it is transmitted via MQTT to AWS IoT Core; then routing the message on AWS IoT Core to an AWS Timestream database; and a browser-based customer-facing dashboard periodically checks the AWS Timestream database to display the contents of the database.

[0012] Some embodiments provide a method for determining an activity of interest related to livestock health and / or welfare, including: for each of a plurality of sensor and processor pairs: receiving, from the sensor, a signal indicative of sensor-perceived input related to the livestock; performing, by the processor, a transform on the received signal; performing, by the processor, neural network processing of the activity of interest on the transformed signal to determine whether an activity of interest has occurred, where the activity of interest has occurred if the perceived input indicates poor health and / or poor welfare of the livestock; and transmitting an indication of the activity of interest to a server.

[0013] Some embodiments also provide filtering the signal indicative of sensor-perceived input related to the livestock, and wherein the processor performs a transform on the filtered received signal.

[0014] Some embodiments also provide binning the transformed signal, and wherein the processor performs neural network processing of the activity of interest on the binned transformed signal.

[0015] In some embodiments, the sensor is an audio sensor. In some embodiments, the sensor is an image sensor. In some embodiments, the sensor is a temperature sensor. In some embodiments, the sensor is an aerosol sensor. In some embodiments, the sensor is an infrared sensor.

[0016] In some embodiments, the transform is a short-time Fourier transform (STFT), and binning of the transformed signal pools the results of the STFT into frequency bins.

[0017] In some embodiments, machine learning training is performed using information indicative of poor health and / or poor welfare of livestock. In some embodiments, the sensor is a sound sensor, and machine learning training is performed using information indicative of the sound of poor health and / or poor welfare of livestock.

[0018] Some embodiments also provide filtering the received signal, wherein the processor performs a transformation on the filtered received signal.

[0019] In some embodiments, the noise filter neural network performs filtering of the received signal by operating on the received signal.

[0020] In some embodiments, the noise filter neural network determines the filtering parameters of the filter to filter the received signal. Some embodiments also provide performing machine learning training on a server to train the noise filter neural network to determine filter parameters to filter the ambient sound in a livestock farm or pen; and downloading the trained noise filter neural network to the processor of the sensor and processor pair.

[0021] In some embodiments, machine learning training of the noise filter neural network is performed using the ambient sound in a livestock farm or pen.

[0022] In some embodiments, the livestock are pigs, and the sensor and processor pairs are distributed around the pig farm. In some embodiments, each pigpen provides at least one sensor and processor pair.

[0023] In some embodiments, the livestock are cattle, and the sensor and processor pairs are distributed around the pasture used by the cattle. In some embodiments, the livestock are cattle, and a sensor and processor pair are provided to each cow or bull.

[0024] Some embodiments provide a system for determining an activity of interest related to livestock health and / or welfare, comprising: a plurality of network edge devices, each of the network edge devices including a sensor and a processor, the processor being configured to perform a transformation on the sensor output, perform neural network processing of the activity of interest on the transformed sensor output to determine the occurrence of an activity of interest related to livestock health and / or welfare, and command the transmission of an indication of the occurrence of the activity of interest to a remote computing device.

[0025] In some embodiments, the network edge device further includes a filter for filtering the sensor output. In some embodiments, the network edge device further includes network communication circuitry. In some embodiments, the transform is a short-time Fourier transform (STFT). In some embodiments, the processor is further configured to bin the results of the STFT, wherein the processor is configured to perform active neural network processing of interest on the binned results of the STFT. In some embodiments, the processor is configured to perform active neural network processing of interest using a neural network trained to identify signals indicative of poor health and / or poor welfare of livestock. In some embodiments, the sensor is an audio sensor. In some embodiments, the sensor is an image sensor. In some embodiments, the sensor is a temperature sensor. In some embodiments, the sensor is an aerosol sensor. In some embodiments, the sensor is an infrared sensor. In some embodiments, the signal indicative of a diseased livestock includes a sound indicative of poor health and / or poor welfare of the livestock. In some embodiments, the network edge device further includes a filter for filtering the sensor output. In some embodiments, the processor includes a first processing chain and a second processing chain. The first processing chain includes a first short-time Fourier transform (STFT) module, a first binning module, and an active neural network of interest, and is arranged to operate on the signal provided by the filter in sequence. The second processing chain includes a second STFT module, a second binning module, and a network filtering neural network, and is also arranged to operate on the signal provided by the filter in sequence.

[0026] These and other aspects of the invention will be more fully understood after reading this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a semi-block diagram of a system for determining activities of interest related to livestock health and / or welfare using an edge device equipped with a neural network according to aspects of the present invention.

[0028] Figure 2 is a flowchart of a process including determining activities of interest related to livestock health and / or welfare according to aspects of the present invention.

[0029] Figure 3 is a flowchart of a process for determining activities of interest in hearing according to aspects of the present invention.

[0030] Figure 4a is a flowchart of a process for training a neural network for a noise filter according to aspects of the present invention.

[0031] Figure 4b is a flowchart of a process for training a neural network for activities of interest in hearing according to aspects of the present invention.

[0032] Figure 5 It is a flowchart of a further process for determining an auditory activity of interest according to an aspect of the present invention.

[0033] Figure 6 It is a flowchart of a process for determining filter parameters according to an aspect of the present invention.

[0034] Figure 7 It is a flowchart of a process for determining parameters of an auditory activity of interest according to an aspect of the present invention.

[0035] Figure 8 It is a block diagram of an example network edge device according to an aspect of the present invention.

[0036] Figure 9 It is a screenshot showing an example activity report of interest according to an aspect of the present invention. Detailed Description

[0037] Figure 1 It is a semi-block diagram of a system for determining an activity of interest related to livestock health and / or welfare using an edge device equipped with a neural network. In some embodiments, the activity of interest is an auditory activity of interest. In some embodiments, the activity of interest is a visual activity of interest. In some embodiments, the activity of interest is a temperature activity of interest. In some embodiments, the activity of interest is an aerosol activity of interest. In some embodiments, the activity of interest is a visual activity in the infrared region. The system includes a plurality of network edge devices 111a-n. The edge device includes a sensor and a processor. In some embodiments, the processor may be replaced by an ASIC or other circuitry. In some embodiments, the sensor is a sound sensor. In various embodiments, the sensor provides a digital output indicative of an input received and processed by the sensor. In various embodiments, the sensor is a sound sensor, and the sound sensor provides a digital output indicative of a sound received by the sound sensor. In some embodiments, the sound sensor may include a MEMS microphone and circuitry for converting a signal generated by the MEMS microphone into a digital signal. In various embodiments, the sensor may be another type of sensor, such as a temperature sensor or other sensors.

[0038] The processor is configured by program instructions to process the output of a sensor. In some embodiments, the processor is configured to perform a transformation on the sensor output, perform an activity neural network of interest processing on the transformed information to determine an activity of interest, and command the transmission of an indication of the activity of interest to a remote server or computing device. In some embodiments, the processor is configured to transform the output of the sensor from the time domain to the frequency domain. In some embodiments, the processor is configured to perform a short-time Fourier transform (STFT) on the output of the sensor. In some embodiments, the processor is configured to form bins, such as based on the results of the frequency-based STFT. In some embodiments, the processor uses the bins to perform an activity neural network of interest processing to determine the presence of an auditory activity of interest.

[0039] In some embodiments, the activity neural network of interest training is performed by server 117. The server is shown in Figure 1 to be accessible via network or “cloud” 115, but in some embodiments, the server may be considered part of the “cloud”. Although only one server is shown, multiple servers may be used in operation. Using a server to train a neural network may be beneficial because the server may have significantly greater computing resources compared to network edge devices, which in some embodiments may be considered Internet of Things (IoT) devices. In some embodiments, the training includes providing an activity neural network of interest input representative of sounds associated with poor livestock health and / or poor livestock welfare. In some embodiments, the training includes providing an activity neural network of interest input representative of sounds associated with poor livestock health and / or poor livestock welfare. For example, for various animals (such as pigs and cows), the sounds of various diseases and ailments, such as coughs, have been identified and / or cataloged. Similarly, sounds indicating poor livestock welfare have been similarly identified. After training, the activity neural network of interest can be loaded onto a network edge device.

[0040] In some embodiments, the network edge device also includes a filter for filtering the output of the sensor. In some of such embodiments, the network edge device may include a noise filtering neural network for determining the settings or parameters of the filter. In such an embodiment, the noise filtering neural network may be provided with the binned STFT output of the audio sensor, and the noise filtering neural network determines the settings or parameters of the filter. The settings or parameters may be in the form of weights or coefficients for an IIR or FIR filter, for example. In some embodiments, the settings or parameters may be used by the ADC circuit of the audio sensor.

[0041] The noise filtering neural network can also be trained on a server. In some embodiments, the noise filtering neural network can be trained using binned STFT information generated from ambient sounds obtained from a location of interest or a location similar to the location of interest. The location of interest can be, for example, a pigsty where pigs are raised in a pig farm, a pasture where cows are grazing, etc. Thus, the noise filtering neural network can be trained to determine filter settings to reduce the impact of ambient noise on the information provided to the active neural network of interest. Like the active neural network of interest, the noise filtering neural network can be loaded onto a network edge device.

[0042] In some embodiments, the active neural network of interest and the noise filtering neural network can be downloaded from the server to the network edge device. In some of such embodiments, the neural network can pass through a router or other network device 113, and the router or other network device 113 provides data communication between the network edge device and the Internet 115.

[0043] In some embodiments, the network edge devices can be distributed around a pig farm, with one or more network edge devices in each pigsty. In some embodiments, the network edge devices can be distributed around a pasture used by cows. In some embodiments, each cow or bull can be provided with its own network edge device.

[0044] In operation, the network edge devices monitor the conditions in the area around the livestock, such as audio conditions. When monitoring the conditions, the network edge devices use their active neural networks of interest to determine whether an activity of interest has occurred. If so, the network edge devices provide a report of the activity of interest, for example, the report is provided to a server that maintains data on the occurrence of the activity of interest. In some embodiments, the server can be accessed via the Internet, and in some embodiments, the server can be considered a "cloud" server. The report on the occurrence of the activity of interest can be viewed by a farmer or rancher on a display of a computing device (such as a laptop 119), so that the farmer or rancher can be alerted to the activity and take corrective measures.

[0045] Figure 2 is a flowchart of a process that includes determining an auditory activity of interest related to the health and / or welfare of livestock. In some embodiments, the process is at least partially performed by a system, such as Figure 1 the system. In some embodiments, the process is at least partially performed by multiple processors. In some embodiments, the process is performed by a network edge device, at least one server, and at least one computing device (such as a laptop, personal computer, tablet, or smartphone).

[0046] In block 211, the process performs machine learning training of at least one neural network. In some embodiments, the training is provided by a server or other computing device accessible via the Internet. In some embodiments, the server or other computing device has significantly greater computing power than the network edge device. In some embodiments, the server or other computing device includes an operating system that supports a machine learning training package, while the network edge device does not. In some embodiments, the neural network is a neural network of interest in an activity. In some embodiments, the neural network of interest in an activity is a convolutional neural network. In some embodiments, the neural network includes at least one intermediate layer. In some embodiments, the neural network includes multiple intermediate layers. In some embodiments, at least one neural network includes multiple neural networks. In some embodiments, the multiple neural networks include a noise filtering neural network. In some embodiments, information indicating poor health and / or welfare of livestock is used to perform the training of the neural network of interest in an activity. In some embodiments, information indicating the sound of poor health and / or welfare of livestock is used to perform the training of the neural network of interest in an activity. In some embodiments, information on the ambient sound of a livestock ranch or pen is used to perform the training of the noise filtering neural network. In some embodiments, the information on the ambient sound of the ranch or pen includes information on the sound of livestock on the ranch or in the pen.

[0047] In block 213, the process downloads the trained neural network to the network edge device. In some embodiments, both the server or other computing device of the process and the network edge device are connected to the Internet, and the process downloads the neural network via the Internet. In some embodiments, the network edge device includes at least one sensor and at least one processor, and the neural network is downloaded to the processor of the network edge device. The sensor can be a sound sensor. In some embodiments, the activity of interest is an auditory activity of interest. In some embodiments, the auditory activity of interest is a specific auditory activity generated by livestock. In some embodiments, the activity of interest is a vocalization activity of interest. In some embodiments, the vocalization activity of interest is a specific vocalization activity of livestock.

[0048] In block 215, the process monitors activities of interest. In most embodiments, a network edge device is used to monitor activities of interest. In some embodiments, the network edge device is placed inside or around a livestock pen, and the process monitors activities of interest in the livestock pen. In some embodiments, the network edge device is placed on the livestock, and the process monitors activities of interest from the livestock. In some embodiments, a sound sensor of the network edge device receives sounds, such as sounds from livestock, and converts the sounds into digital signals. The digital signals can be filtered, for example, using a filter of the network edge device. In some embodiments, the parameters of the filter can be determined by a noise filter neural network of the network edge device. The filtered signal can be transformed into the frequency domain, for example, using the STFT executed by a processor of the network edge device. The transformed signal can be used to generate frequency bins by the network edge device processor, and the information thereof is provided to a neural network of activities of interest executed on the processor. The frequency bins can be used to reduce the number of inputs to the neural network.

[0049] In block 217, the process determines whether an auditory activity of interest has occurred. In most embodiments, the process determines whether an auditory activity of interest has occurred based on the output of the neural network of activities of interest. If no auditory activity of interest has occurred, the process continues to monitor in block 215. Otherwise, the process proceeds to block 219.

[0050] In block 219, the process reports the result of the monitoring. In most embodiments, the result is that an auditory activity of interest is determined to have occurred. In some embodiments, the reported result is a specific auditory activity of interest, which is one of potentially multiple auditory activities of interest. In some embodiments, the result is reported by the network edge device. In some embodiments, the network edge device transmits a message indicating the result on the network. In some embodiments, the network includes the Internet. In some embodiments, the result is transmitted to a server accessible via the Internet.

[0051] In block 221, the process displays the result. In some embodiments, the result is displayed on a computing device of a farmer or rancher associated with the livestock being monitored. In some embodiments, the result includes an identification of the activity of interest and the location of the network edge device reporting the result.

[0052] In optional block 223, corrective measures regarding the livestock are performed. The corrective measures can be performed by or at the request of a farmer or rancher.

[0053] Thereafter, the process returns. In some embodiments, the process can return to block 215 and continue the monitoring operation.

[0054] Figure 3is a flowchart of a process for determining an auditory activity of interest. In some embodiments, the process is performed by a system, such as Figure 1 the system. In some embodiments, the process is performed by a network edge device, such as Figure 1 the network edge device. In some embodiments, the process is performed at least in part by one or more processors. In some embodiments, Figure 3 the process performs Figure 2 the operations of blocks 215-219 of the process.

[0055] In block 311, the process digitizes an audio signal. In some embodiments, the digitization of the audio signal is performed by circuitry of a sound sensor of the network edge device or circuitry associated therewith.

[0056] In block 313, the process transforms the digitized audio signal into the frequency domain. In some embodiments, the process uses an STFT to transform the digitized signal. In some embodiments, the transformation is performed by a processor of the network edge device. In some embodiments, the process also stores information of the transformed signal in frequency bins.

[0057] In block 315, the process performs neural network processing of the transformed signal. In some embodiments, the process performs neural network processing of the binned transformed signal. In most embodiments, the process uses a neural network of interest to perform the neural network processing. In some embodiments, the neural network processing is performed by a neural network of interest executing on a processor of the network edge device.

[0058] In block 317, the process determines whether an activity of interest has occurred. In some embodiments, if the output of the neural network of interest indicates that an activity of interest has occurred, the process determines that an activity of interest has occurred. If so, the process continues with the operation of block 319, otherwise the process returns.

[0059] In block 319, the process reports the activity of interest. In some embodiments, the network edge device transmits a message indicating that an activity of interest has occurred. In some embodiments, the message indicates the time of the activity of interest and the type of the activity of interest.

[0060] Thereafter, the process returns.

[0061] Figure 4a is a flowchart of a process for training a noise filter neural network. In some embodiments, the process is performed at least in part by a system, such as Figure 1 the system. In some embodiments, the process is performed at least in part by at least one processor. In some embodiments, the process is performed at least in part by a server. In some embodiments, Figure 4aProcess execution Figure 2 Operation of block 211 of the process

[0062] In block 411, environmental sounds of a livestock pen or pasture are recorded or obtained. In some embodiments, the livestock pen or pasture houses livestock. In some embodiments, the environmental sounds are digitized, transformed into the frequency domain, and binned.

[0063] In block 413, the process uses the information of the environmental sounds to train a neural network. In some embodiments, the neural network is trained to determine filter parameters or coefficients for filtering environmental sounds in a herd pasture or pen. In some embodiments, the neural network thus trained is a noise filter neural network.

[0064] Then the process returns.

[0065] Figure 4b Is a flowchart of a process for training a neural network for an auditory activity of interest. In some embodiments, the process is performed at least in part by a system, such as Figure 1 a system. In some embodiments, the process is performed at least in part by at least one processor. In some embodiments, the process is performed at least in part by a server. In some embodiments, Figure 4b Process execution Figure 2 Operation of block 211 of the process

[0066] In block 421, an auditory activity of interest is recorded or obtained. In some embodiments, the auditory activity of interest is a sound indicating livestock disease and / or poor livestock welfare. In some embodiments, the auditory activity of interest is digitized, transformed into the frequency domain, and binned.

[0067] In block 423, the process uses the information of the auditory activity of interest to train a neural network. In some embodiments, the neural network is trained to determine whether an auditory activity of interest has occurred. In some embodiments, the neural network thus trained is an activity-of-interest neural network.

[0068] Then the process returns.

[0069] Figure 5 Is a flowchart of a further process for determining an auditory activity of interest. Figure 5 The process includes filter processing. In some embodiments, the process is performed by a system, such as Figure 1 a system. In some embodiments, the process is performed by a network edge device, such as Figure 1 a network edge device. In some embodiments, the process is performed at least in part by one or more processors. In some embodiments, the process is performed by a processor of a network edge device. In some embodiments, Figure 3Process execution Figure 2 Operations of blocks 215 - 219 of the process.

[0070] In block 511, the process performs filtering processing. In some embodiments, the process performs the filtering process by providing sound information to a noise filter neural network and causing the noise filter neural network to operate based on the sound information. In some embodiments, the sound information is information of sound generated by a sound sensor, which has been transformed into the frequency domain (e.g., using STFT) and binned. In some embodiments, the transformation and binning of the sound information are performed as part of the filtering process. In some embodiments, the noise filter neural network determines filter parameters using the sound information. In some embodiments, the sound sensor is a sound sensor of a network edge device, and the noise filter neural network is executed on a processor of the network edge device.

[0071] In block 513, the process sets the filtering parameters of a filter using the filter parameters determined by the noise filter neural network. In some embodiments, the filter is a filter of a network edge device. In some embodiments, the filter is an IIR filter. In some embodiments, the filter is an FIR filter. In some embodiments, the filter is associated with an ADC of the sound sensor.

[0072] In block 515, the process performs activity - of - interest processing. In some embodiments, the process performs the activity - of - interest processing by providing sound information to an activity - of - interest neural network and causing the activity - of - interest neural network to operate based on the sound information. In some embodiments, the sound information is information of sound generated by a sound sensor, which has been filtered by the filter, transformed into the frequency domain (e.g., using STFT) and binned. In some embodiments, the transformation and binning of the sound information are performed as part of the activity - of - interest processing. In some embodiments, the activity - of - interest neural network is executed on a processor of the network edge device.

[0073] In block 517, the process determines whether an activity of interest has occurred. In some embodiments, if the output of the activity - of - interest neural network indicates that an activity of interest has occurred, the process determines that an activity of interest has occurred. If so, the process proceeds to the operation of block 519; otherwise, the process returns.

[0074] In block 519, the process reports the activity of interest. In some embodiments, the network edge device sends a message indicating that an activity of interest has occurred. In some embodiments, the message indicates the time of the activity of interest and the type of the activity of interest.

[0075] Then the process returns.

[0076] Figure 6 is a flowchart of a process for determining filter parameters. In some embodiments, the process is performed by a system, such as Figure 1 a system. In some embodiments, the process is performed by a network edge device, such as Figure 1 a network edge device. In some embodiments, the process is performed at least in part by one or more processors. In some embodiments, the process is performed by a processor of a network edge device. In some embodiments, the process performs Figure 5 the operations of blocks 511 and 513 of the

[0077] In block 611, the process performs an STFT on an input signal. In some embodiments, the input signal is the output of an audio signal. In some embodiments, the input signal is the digital output of a sound sensor. In some embodiments, the sound sensor is a sound sensor of a network edge device.

[0078] In block 613, the process bins the transformed signal. In some embodiments, the binning is frequency binning.

[0079] In block 615, the transformed and binned signal is operated on by a noise filter neural network. The noise filter neural network determines filter parameters.

[0080] In block 617, the filter parameters are provided to a filter.

[0081] Then the process returns.

[0082] Figure 7 is a flowchart of a process for determining parameters of an auditory activity of interest. In some embodiments, the process is performed by a system, such as Figure 1 a system. In some embodiments, the process is performed by a network edge device, such as Figure 1 a network edge device. In some embodiments, the process is performed at least in part by one or more processors. In some embodiments, the process is performed by a processor of a network edge device. In some embodiments, the process performs Figure 5 the operation of block 515 of the

[0083] In block 711, the process filters an input signal. In some embodiments, the filter has parameters set by the Figure 6 process. In some embodiments, the input signal is the output signal of a sound sensor.

[0084] In block 713, the process performs an STFT on the filtered signal.

[0085] In block 613, the process bins the filtered and transformed signal. In some embodiments, the binning is frequency binning.

[0086] In block 615, the filtered, transformed, and binned signals are operated on by the active neural network of interest. The active neural network of interest determines whether an activity of interest has occurred.

[0087] The process then returns.

[0088] Figure 8 is a block diagram of an exemplary network edge device. The exemplary network edge device can be a Figure 1 network edge device. The network edge device includes a sensor 811 and a processor 813. Additionally, the network edge device can include network communication capabilities (not shown in Figure 8 ) such as one or more of Bluetooth circuitry, Wi-Fi circuitry, Ethernet circuitry, or other communication circuitry.

[0089] The sensor can be a sound sensor. The sound sensor can include a filter for filtering the output of the sound sensor, or the filter can be provided as a separate component of at least a portion of the network edge device. Figure 8 A separate filter 831 is shown, although in some embodiments, some portions of the filter can be implemented as part of the processor and other portions (such as inductive elements) can be implemented as a part separate from the processor.

[0090] The processor includes two processing chains. The first processing chain includes an STFT module 821b, a binning module 823b, and an active neural network of interest 825b, all arranged to operate sequentially on the signal provided by the filter. The output of the active neural network of interest can be transmitted over the network. In some embodiments, the processor also includes logic, such as commanding the transmission of a message indicating the output of the active neural network of interest when the neural network indicates that an activity of interest has occurred. The second processing chain includes a further STFT module 821a, a further binning module 823a, and a noise filter neural network 825a, all arranged to operate sequentially on the signal provided by the sound sensor. The output of the noise filter neural network can be used to set the filtering parameters of the filter. In some embodiments, the operations of the first processing chain or the operations of the active neural network of interest are not performed until after the operations of the second processing chain have set the filtering parameters of the filter. In some embodiments, a register or other signal is set to indicate when the operations of the noise filter neural network have provided an output for setting the filter parameters, and the register or other signal is used to indicate that the operations of the first processing chain or the active neural network of interest can be performed. In some embodiments, a register or other signal is used to indicate that the output of the active neural network of interest is valid. In some embodiments, the use of the register or other signal can reduce the number of false positives from the active neural network of interest.

[0091] Figure 9 It is a screenshot showing an example activity report of interest. The screenshot can be of a computing device, such as Figure 1 a screenshot of the display of a computing device. The screenshot shows a column labeled "Activity" 911, a column labeled "Time" 913, and a column labeled "Additional Information" 915. The Activity column can include information indicating activities of interest detected by a network edge device (such as Figure 1 a network edge device). The Time column can indicate the time of occurrence of the activity of interest. The additional information can include additional information, such as further information about the activity of interest and / or information about the network edge device reporting the activity, such as the location of the network edge device.

[0092] Although the present invention has been discussed with reference to various embodiments, it should be recognized that the present invention includes novel and non-obvious claims supported by this disclosure.

Claims

1. A method for determining an activity of interest related to livestock health and / or welfare, comprising: For each of a plurality of sensor and processor pairs: Receiving, from the sensor, a signal indicative of livestock-related input sensed by the sensor; Performing, by the processor, a transformation on the received signal; Performing, by the processor, neural network processing of the activity of interest on the transformed signal to determine whether the activity of interest has occurred, where the activity of interest occurs if the sensed input indicates poor health and / or poor welfare of the livestock; And Sending an indication of the activity of interest to a server.

2. The method according to claim 1, further comprising filtering the signal indicative of livestock-related input sensed by the sensor, and wherein the processor performs a transformation on the received filtered signal.

3. The method according to claim 2, further comprising binning the transformed signal, and wherein the processor performs neural network processing of the activity of interest on the binned transformed signal.

4. The method according to claim 3, wherein the sensor is an audio sensor.

5. The method according to claim 3, wherein the sensor is an image sensor.

6. The method according to claim 3, wherein the sensor is a temperature sensor.

7. The method according to claim 3, wherein the sensor is an aerosol sensor.

8. The method according to claim 3, wherein the sensor is an infrared sensor.

9. The method according to claim 3, wherein the transformation is a short-time Fourier transform (STFT), and binning of the transformed signal pools the results of the STFT into frequency bins.

10. The method according to any one of the preceding claims, wherein each sensor and processor pair is a network edge device.

11. The method according to any one of the preceding claims, further comprising: Performing machine learning training on the server to train a neural network to identify an activity of interest of the livestock, the activity of interest being poor health and / or poor welfare of the livestock; And Downloading the trained neural network to the processor of the sensor and processor pair.

12. The method according to claim 11, wherein information indicative of poor health and / or poor welfare of the livestock is used to perform the machine learning training.

13. The method according to claim 11, wherein the sensor is a sound sensor, and information indicative of sounds of poor health and / or poor welfare of the livestock is used to perform the machine learning training.

14. The method according to any one of the preceding claims, further comprising filtering the received signal, and wherein the processor performs a transformation on the received filtered signal.

15. The method according to claim 14, wherein a noise filter neural network performs filtering of the received signal by operating on the received signal.

16. The method according to claim 14, wherein the noise filter neural network determines filtering parameters of the filter to filter the received signal.

17. The method according to claim 16, further comprising: Perform machine learning training on a server to train a noise filter neural network to determine filtering parameters for filtering environmental sounds in a livestock farm or pen; and Download the trained noise filter neural network to the processor of a sensor and processor pair.

18. The method according to claim 17, wherein the machine learning training of the noise filter neural network is performed using environmental sounds in a livestock farm or pen.

19. The method according to any one of the preceding claims, wherein the livestock are pigs, and wherein the sensor and processor pairs are distributed around a pig farm.

20. The method according to claim 19, wherein each pigsty provides at least one sensor and processor pair.

21. The method according to any one of claims 1 to 18, wherein the livestock are cattle, and wherein the sensor and processor pairs are distributed around a pasture used by the cattle.

22. The method according to any one of claims 1 to 18, wherein the livestock are cattle, and wherein a sensor and processor pair is provided to each cow or bull.

23. A system for determining an activity of interest related to livestock health and / or welfare, comprising: A plurality of network edge devices, each of the network edge devices including a sensor and a processor, the processor being configured to perform a transformation on the sensor output, perform an activity-of-interest neural network process on the transformed sensor output to determine the occurrence of an activity of interest related to livestock health and / or welfare, and command the transmission of an indication of the occurrence of the activity of interest to a remote computing device.

24. The system according to claim 23, wherein the network edge device further includes a filter for filtering the output of the sensor.

25. The system according to claim 23, wherein the network edge device further includes network communication circuitry.

26. The system according to claim 23, wherein the transformation is a short-time Fourier transform (STFT).

27. The system according to claim 26, wherein the processor is further configured to bin the result of the STFT, and the processor is configured to perform an activity-of-interest neural network process on the binned result of the STFT.

28. The system according to claim 23, wherein the processor is configured to perform an activity-of-interest neural network process using a neural network trained to identify signals indicating poor health and / or poor welfare of the livestock.

29. The system according to claim 28, wherein the sensor is an audio sensor.

30. The method according to claim 28, wherein the sensor is an image sensor.

31. The method according to claim 28, wherein the sensor is a temperature sensor.

32. The method according to claim 28, wherein the sensor is an aerosol sensor.

33. The method according to claim 28, wherein the sensor is an infrared sensor.

34. The system according to claim 29, wherein the signal indicating a diseased livestock includes a sound indicating poor health and / or poor welfare of the livestock.

35. The system according to claim 23, wherein the network edge device further comprises a filter for filtering the output of the sensor.

36. The system according to claim 35, wherein the processor comprises a first processing chain and a second processing chain. The first processing chain comprises a first short-time Fourier transform (STFT) module, a first binning module, and an active neural network of interest, and is arranged to operate on the signal provided by the filter in sequence. The second processing chain comprises a second STFT module, a second binning module, and a network filter neural network, and is also arranged to operate on the signal provided by the filter in sequence.

Images