Detecting functional impairment of camera-monitored insect traps

Abstract

The systems, methods and computer programs disclosed by the invention relate to the automated detection of functional impairment of camera-monitored insect traps by way of machine learning methods.

Description

[0001] Detecting functional impairments in camera-monitored insect traps

[0002] TECHNICAL FIELD

[0003] The systems, methods, and computer programs disclosed herein relate to the automated detection of malfunctions in camera-monitored insect traps using machine learning methods.

[0004] INTRODUCTION

[0005] WO2018054767A1 discloses an insect trap that a user visits to check whether any insects have entered the trap. Using a smartphone, the user captures an image of the insects in the trap. A computer program automatically detects, counts, and / or identifies the insects depicted in the image.

[0006] W02020058175A1 and W02020058170A1 disclose an insect trap equipped with a camera. The camera automatically captures images of a collection area of ​​the insect trap where insects are gathering. The images are transmitted via a transmitter unit to a separate computer system, where they are reviewed by a user or analyzed using image recognition algorithms to determine the number of insects in the collection area and / or to identify the insects.

[0007] The insect traps disclosed in W02020058175A1 and W02020058170A1 have the advantage over the insect trap disclosed in WO2018054767A1 that the user does not have to visit the insect traps to check whether insects have entered the insect traps.

[0008] It is possible that a camera-monitored insect trap may experience a functional impairment over time. For example, a liquid used in the insect trap to immobilize insects may evaporate over time. The liquid level in the insect trap may drop to a level where insects are no longer immobilized. Another possible functional impairment could be contamination of the collection area and / or optical elements of the camera (e.g., a lens). Other possible functional impairments are listed later in the description.

[0009] A functional impairment means that the function intended to be fulfilled by the insect trap is no longer fulfilled or no longer fulfilled sufficiently.

[0010] SUMMARY

[0011] This problem is solved by the subject matter of the independent claims of the present disclosure. Preferred embodiments can be found in the dependent claims, the description, and the drawings.

[0012] The present disclosure describes means by which a functional impairment of a camera-monitored insect trap can be detected at an early stage.

[0013] A first aspect of the present disclosure is a computer-implemented method for detecting a malfunction in a camera-monitored insect trap. The detection method comprises:

[0014] Receiving an image recording, wherein the image recording shows at least part of a collection area of ​​an insect trap, providing a trained machine learning model, wherein the machine learning model is configured and has been trained on the basis of training data to distinguish image recordings of insect traps with a functional impairment from image recordings of insect traps without functional impairment,

[0015] Feeding the image capture to the trained machine learning model,

[0016] Receiving information from the machine learning model, the information indicating whether the image recording shows an insect trap with a functional impairment, in the event that the image recording shows an insect trap with a functional impairment: issuing a notification that the insect trap shown at least partially in the image recording has a functional impairment.

[0017] A further subject of the present disclosure is a computer system comprising: an input unit, a control and computing unit, and an output unit, wherein the control and computing unit is configured to cause the input unit to receive an image recording, wherein the image recording shows at least part of a collection area of ​​an insect trap, to feed the received image recording to a machine learning model, wherein the machine learning model is configured and has been trained on the basis of training data to distinguish image recordings of insect traps with a functional impairment from image recordings of insect traps without a functional impairment, to receive information from the machine learning model, wherein the information indicates whether the received image recording shows an insect trap with a functional impairment, to cause the output unit to output a message,wherein the notification indicates that the insect trap shown at least partially in the received image has a functional impairment if the information output by the machine learning model indicates that the received image shows an insect trap with a functional impairment.

[0018] Another subject of the present disclosure is a non-transitory computer-readable storage medium having stored thereon software instructions that, when executed by a processor of a computer system, cause the computer system to perform the following steps:

[0019] Receiving an image recording, wherein the image recording shows at least part of a collection area of ​​an insect trap,

[0020] Feeding the received image to the trained machine learning model, wherein the machine learning model is configured and trained on the basis of training data to distinguish images of insect traps with a functional impairment from images of insect traps without a functional impairment,

[0021] Receiving information from the machine learning model, the information indicating whether the received image recording shows an insect trap with a functional impairment, in the event that the information indicates that the received image recording shows an insect trap with a functional impairment: issuing a notification that the insect trap shown at least partially in the received image recording has a functional impairment.

[0022] Another subject of the present disclosure is a system comprising:

[0023] • an insect trap,

[0024] • a camera,

[0025] • a control unit,

[0026] • an analysis unit,

[0027] • a transmitting unit, and

[0028] • an output unit, wherein the control unit is configured to cause the camera to generate one or more images of a collection area of ​​the insect trap, wherein the analysis unit is configured to feed the one or more generated images to a machine learning model, wherein the machine learning model is configured and has been trained on the basis of training data to distinguish images of insect traps with a functional impairment from images of insect traps without a functional impairment, wherein the analysis unit is configured to receive information from the machine learning model, wherein the information indicates whether the one or more images show / show an insect trap with a functional impairment, wherein the transmission unit is configured to transmit the information and / or the one or more images to a separate computer system,wherein the output unit is configured to output the information.,

[0029] A further subject of the present disclosure is a kit comprising an insect trap and a computer program product, wherein the insect trap comprises a camera or means for receiving a camera, wherein the computer program product comprises program instructions, wherein the program instructions can be loaded into a working memory of a computer system and cause the computer system to perform the following steps:

[0030] Receiving an image recording, wherein the image recording shows at least part of a collection area of ​​an insect trap,

[0031] Feeding the received image to the trained machine learning model, wherein the machine learning model is configured and trained on the basis of training data to distinguish images of insect traps with a functional impairment from images of insect traps without a functional impairment,

[0032] Receiving information from the machine learning model, the information indicating whether the received image shows an insect trap with a functional impairment. If the information indicates that the received image shows an insect trap with a functional impairment, outputting a notification that the insect trap shown at least partially in the received image has a functional impairment. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Fig. 1 shows schematically and exemplarily the training of a machine learning model.

[0034] Fig. 2 schematically shows the use of a trained machine learning model to detect a functional impairment in an insect trap.

[0035] Fig. 3 schematically shows another example of training a machine learning model.

[0036] Fig. 4 schematically shows another example of using a trained machine learning model to detect a functional impairment in an insect trap.

[0037] Fig. 5 shows an example and schematically a computer-implemented method for training a machine learning model in the form of a flow chart.

[0038] Fig. 6 shows an example and schematically a computer-implemented method for detecting a functional impairment in a camera-monitored insect trap in the form of a flow chart.

[0039] Fig. 7 shows an exemplary and schematic illustration of an embodiment of a computer system of the present disclosure.

[0040] Fig. 8 shows an exemplary and schematic illustration of another embodiment of a computer system of the present disclosure.

[0041] DETAILED DESCRIPTION

[0042] The invention is explained in more detail below, without distinguishing between the subject matters of the present disclosure (method, computer system, computer-readable storage medium, system, kit). Rather, the following statements apply mutatis mutandis to all subject matters of the invention, regardless of the context in which they are described (method, computer system, computer-readable storage medium, system, kit).

[0043] If steps are specified in a particular order in this description or in the claims, this does not necessarily mean that the invention is limited to the specified order. Rather, it is conceivable that the steps could also be performed in a different order or even in parallel, unless one step builds on another, which necessarily requires that the subsequent step be performed (although this will become clear in the individual case). The specified orders are therefore preferred embodiments of the present disclosure.

[0044] The invention is explained in more detail at some points with reference to drawings. The drawings depict specific embodiments with specific features and combinations of features, which primarily serve for illustrative purposes; the invention should not be understood as being limited to the features and combinations of features shown in the drawings. Furthermore, statements made in the description of the drawings regarding features and combinations of features are intended to apply generally, meaning they are also transferable to other embodiments and are not limited to the embodiments shown.

[0045] The present disclosure describes means for detecting early and automatically a malfunction in a camera-monitored insect trap.

[0046] An insect trap is a device that is randomly or deliberately visited by insects and that allows a user to detect whether insects are present in an area. The area can be, for example, a field or a greenhouse for growing crops, a building (e.g., a storage room for storing food and / or animal feed, a hospital, a retirement home, a school, and / or the like), a room within a building (e.g., a living room, a bedroom, a cafeteria, a hospital ward, and / or the like), or another area.

[0047] The term insect includes all stages from the larva (caterpillar, dew-worm) to the adult stage.

[0048] The term "insect trap" should not be understood to imply that only insects can enter it. The term "insect trap" was chosen because such traps are primarily used to check whether insects are present in an area. However, the insect trap can also be used to detect the presence of other arthropods (Latin: Arthropoda), such as spiders. It is also conceivable that the insect trap is used to detect the presence of a specific species in an area and is accordingly designed to attract, immobilize, and / or detect this specific species.

[0049] Examples of such defined species are: codling moth, aphid, natterjack, fruit skin tortrix, Colorado potato beetle, cherry fruit fly, cockchafer, European corn borer, plum tortrix, rhododendron leafhopper, seed moth, scale insect, gypsy moth, spider mite, grape tortrix, walnut fruit fly, whitefly, large rapeseed stem weevil, spotted cabbage stem weevil, rapeseed pollinator, cabbage pod weevil, cabbage pod midge or rapeseed flea, or a forest pest such as aphid, blue pine jewel beetle, bark beetle, oak jewel beetle, oak processionary moth, oak tortrix, spruce web sawfly, common woodworm, large brown barkeater, pine bush sawfly, pine owl, pine moth, small spruce sawfly, nun moth, Horse chestnut leaf miner, gypsy moth, sapwood beetle, mosquitoes (e.g. Asian tiger mosquito).

[0050] The insect trap may comprise means for immobilizing insects. The insect trap may, for example, comprise a bowl filled with a liquid (e.g., water or an aqueous solution). It is conceivable that the liquid comprises a surfactant to reduce surface tension and / or an anti-algae agent and / or an attractant to attract insects. Insects that enter the liquid may, for example, drown in the liquid or be trapped by the liquid. The insect trap may also comprise a surface coated with glue or another adhesive to which insects stick. However, an insect trap does not have to comprise means for immobilizing insects; it may be sufficient for the intended use of the insect trap if insects enter the collection area and remain there for a while.

[0051] The insect trap may include means for attracting insects. Some insects (e.g., rapeseed pests such as the large rapeseed stem weevil) are attracted by a yellow color, for example. Some insects (e.g., male pantry moths) can be attracted by a pheromone. The use of food for the insects to attract them is also possible. Some insects are attracted by electromagnetic radiation within a defined wavelength range. The insect trap may be equipped with a source of electromagnetic radiation that emits electromagnetic radiation within a defined wavelength range (or multiple wavelength ranges). However, the insect trap does not need to contain any attractants; for the intended use of the insect trap, it may be sufficient for insects to accidentally wander into the trap or come into contact with it.

[0052] The insect trap includes a collection area. The collection area is an area that can be visited by insects (or other arthropods). This can be a flat surface, such as a board or map. It can also be the bottom of a container. The insect trap may include multiple collection areas. It is also conceivable for the insect trap to have different collection areas, for example, one collection area for (specific) pests and another collection area for (specific) beneficial organisms.

[0053] The collection area preferably comprises a flat, smooth or textured surface with a round, oval, elliptical, angular (triangular, quadrangular, pentagonal, hexagonal or generally n-sided, where n is an integer greater than or equal to three) cross-section. The cross-section is preferably round or rectangular (in particular square). Walls can extend upwards from the surface to form a container. The container can, for example, be cylindrical, conical or box-shaped. It preferably has a round or angular cross-section and the walls extend upwards from the base in a conical shape, with the base surface and wall surface preferably running at an angle of more than 90° and less than 120° to one another. In the case of a angular cross-section, the corners can be rounded.

[0054] The bottom of the container may have markings and / or a structure that allows automated focusing of the camera and / or that provides a reference, for example to determine the size of an insect.

[0055] The bottom of the container can have recesses, as described, for example, in WO2022243150A1, to achieve the separation of insects in the collection area. Separation facilitates the automated detection, counting, and / or identification of insects in the collection area.

[0056] The collection area can be part of a pest trapping device, such as a yellow trap tray or an optionally glued color board.

[0057] The insect trap is preferably a trap as described in W02020058175A1, WQ2020058170A1, WO2021213824A1 or WO2022243150A1.

[0058] The insect trap is a camera-monitored insect trap. This means that a camera is positioned and aligned to capture images of a collection area within the insect trap.

[0059] A camera is a device that can create images in digital form and store them and / or make them available via an interface. A camera typically comprises an image sensor and optical elements. The image sensor is a device for electrically capturing two-dimensional images from light. These are typically semiconductor-based image sensors such as CCD (charge-coupled device) or CMOS (complementary metal-oxide-semiconductor) sensors. The optical elements (lenses, apertures, and the like) serve to create the sharpest possible image of the object, of which a digital image is to be created, on the image sensor.

[0060] The camera can, for example, be a component of a smartphone or tablet computer.

[0061] The camera is used to generate digital images of the collection area or a portion thereof. The generated images can be used (i) to detect whether one or more insects are present in the collection area (insect detection), (ii) to count insects in the collection area, and / or (iii) to identify insects, i.e., to determine the type of insect (subclass, superorder, order, suborder, family, genus, species, stage, and / or sex).

[0062] To image the collection area on one or more image sensors, a light source is required to illuminate the collection area so that light (electromagnetic radiation in the infrared, visible, and / or ultraviolet ranges of the spectrum) is scattered / reflected from the illuminated collection area toward the camera. Daylight can be used for this purpose. However, it is also conceivable to use a lighting unit that provides defined illumination independent of daylight. This is preferably mounted to the side of the camera so that the camera does not cast a shadow on the collection area.

[0063] It is also conceivable to position a lighting source below the collection area and / or next to the collection area, which illuminates the collection area "from below" and / or "from the side", while a camera takes one or more images "from above".

[0064] It is conceivable that several lighting sources illuminate the collection area from different directions.

[0065] The terms "light" and "illumination" should not imply that the spectral range is limited to visible light (approximately 380 nm to approximately 780 nm). It is also conceivable that electromagnetic radiation with a wavelength below 380 nm (ultraviolet light: 100 nm to 380 nm) or above 780 nm (infrared light: 780 nm to 1000 pm) is used for illumination. The image sensor and optical elements are usually adapted to the electromagnetic radiation used.

[0066] The camera-monitored insect trap includes a control unit. The control unit can be a component of the camera or a separate device. The control unit is configured to cause the camera to generate one or more images of a collection area or a portion thereof. The control unit can be a computer system, as described later in the description. The camera can be a component of such a computer system.

[0067] The one or more image recordings may be individual image recordings or sequences of image recordings (e.g. video recordings).

[0068] The control unit can be configured to cause the camera to capture images at defined times (e.g., once a day at 12 noon) and / or recurringly at defined time intervals (e.g., every hour between 7 a.m. and 8 p.m.) and / or upon the occurrence of a defined event (e.g., after sunrise when a defined brightness is reached or as a result of the detection of an insect by a sensor) and / or as a result of a command from a user. The control unit can be part of the camera or a device independent of the camera that can communicate with the camera via a wired or wireless connection (e.g., Bluetooth).

[0069] The insect trap further comprises a transmitting unit. Images and / or information can be transmitted to a separate computer system via the transmitting unit. Transmission preferably occurs via a radio network, for example, a mobile network. The transmitting unit can be a component of the control unit or a unit independent of the control unit.

[0070] The insect trap may include a receiving unit to receive commands from a separate computer system.

[0071] The transmitting unit and / or the receiving unit may be components of a computer system as described further down in the description.

[0072] The images generated by the camera are typically analyzed automatically to detect, count, and / or identify the insects present in the collection area or part thereof. This analysis can be performed by an analysis unit that may be part of the insect trap; however, it can also be performed by an analysis unit that may be part of a separate computer system to which the images are transmitted via the transmission unit of the insect trap. The analysis unit may be part of a computer system as described further down in the description. The analysis unit may comprise a trained machine learning model that is configured and trained to detect, count, and / or identify insects depicted in images.Details on the automated detection, counting and / or identification of insects in images are described in publications on this topic (see e.g.: DCK Amarathunga et al. : Methods of Insect Image Capture and Classification: A Systematic Literature Review, Smart Agricultural Technology, Volume 1, 2021, 100023; C. Zhu et al.'. Insect Identification and Counting in Stored Grain: Image Processing Approach and Application Embedded in Smartphones, Mob. Inf. Syst. 2018, 5491706:1-5, W02020058175A1, W02020058170A1).

[0073] The images generated by the camera can also be used to detect any malfunction of the camera-monitored insect trap.

[0074] A functional impairment describes a condition of the insect trap that affects one or more components of the insect trap, or the insect trap as a whole, in such a way that one or more functions are no longer performed, no longer performed sufficiently, or no longer performed optimally. For example, a function may no longer be performed sufficiently or optimally if the impairment slows down or impedes the function, or the result is inferior or faulty.

[0075] Functions that the insect trap is typically required to perform are attracting insects and / or immobilizing insects and / or isolating insects in the collection area and / or illuminating the collection area with one or more light sources and / or generating images of the collection area and / or sending images and / or other / further information to a separate computer system and / or other / further functions.

[0076] A functional impairment can be an impairment that currently affects the function or will affect the function in the near future if no measures are taken to maintain the functions.

[0077] Examples of functional impairments are listed below:

[0078] Liquid level has dropped below a lower threshold: If the collection area of ​​the insect trap is designed as a container, a liquid (e.g., water or an aqueous solution) in the container can be used to immobilize insects. If the amount of liquid in the container drops below a threshold, insects may no longer be immobilized. This represents a functional impairment.

[0079] It is also possible that insects are only partially covered by liquid, making automated detection, counting, and / or identification more difficult. For example, the contours of insects may not be clearly visible in an image due to partial coverage, and / or unwanted reflections may occur at the transition points between the liquid and the insect.

[0080] It is also possible that the liquid has completely evaporated, meaning that the collection area has dried out.

[0081] It is also possible for the insect trap to be equipped with a storage container from which liquid automatically flows into the collection area when the liquid level in the collection area has fallen below a defined threshold (as described, for example, in WO2021213824A1). It is possible for the amount of liquid still contained in the storage container to be shown in an image recording. If the amount of liquid contained in the storage container has fallen below a defined residual amount, it is possible that in the near future no more liquid will be able to flow from the storage container into the collection area; a functional impairment is imminent. Liquid level has risen above an upper threshold: It is possible that, for example, as a result of rainwater, the liquid level in the collection area has risen above an upper threshold.It is possible that liquid may flow uncontrollably over a wall of the container forming the collection area, thereby removing insects floating on the liquid from the collection area.

[0082] Collection area is contaminated: Contaminants may have accumulated in the collection area, making it difficult to detect, count, and / or identify insects. Contaminants may completely or partially obscure insects or clump with them (form agglomerates). Contaminants may include leaves and / or other plant parts, dust, insect and / or other animal droppings, and / or similar items.

[0083] Collection area is exhausted: It is possible that a large number of insects have already collected in the collection area, which are partially or completely overlapping and / or aggregating into clusters. This can impair the automated detection, counting, and / or identification of the insects.

[0084] Algae growth in the collection area: It is possible that algae may form in a liquid in the collection area. This algae can make it difficult to automatically detect, count, and / or identify insects in the collection area.

[0085] Foam in the collection area: Over time, a liquid-filled trap tray can develop foam on top of the liquid. This foam can completely or partially obscure the view of insects in the collection area. It's also possible that insects are no longer immobilized by the liquid.

[0086] Ice formation or icing: If a trap is filled with liquid, the liquid can freeze at low temperatures. This may prevent insects from being immobilized by the liquid.

[0087] Restrictions on the field of view: Spider webs in the insect trap may obstruct the camera's view of the collection area. In addition to cobwebs, spiders in front of a camera lens can also pose a problem. Furthermore, insect constructs (e.g., pupae of larvae) and / or plant parts (e.g., twigs, leaves, roots) in the insect trap may completely or partially obscure the collection area from the camera's perspective.

[0088] Camera and / or optical elements are dirty: Deposits on a camera lens can cause impairment. It is possible that, as a result of deposits, the camera's field of view is restricted, and the entire original collection area is no longer captured. Deposits may result in blurred or partially blurred images. It is possible that water (e.g., rainwater) and / or another liquid (e.g., a liquid from the collection area) may get onto a lens, restricting the field of view and / or causing blurred images.

[0089] Illumination source(s) defective and / or dirty: If the insect trap is equipped with one or more illumination sources, it is possible that one or more of these illumination sources is emitting no or less electromagnetic radiation, and / or that, due to contamination of one or more of the illumination sources, insufficient electromagnetic radiation reaches and illuminates the collection area. The lack of or reduced illumination can lead to a loss of contrast and / or increased noise in the images, which in turn can complicate the automated detection, counting, and / or identification of insects.

[0090] Unwanted reflections: It is possible that reflections can be observed in the images at certain times. These could be caused, for example, by sunlight entering the collection area at a defined angle. It is possible that sunlight enters the collection area at certain times of the day and / or year and causes unwanted reflections there. It is possible that such unwanted reflections from sunlight were not observed when the insect trap was set up and / or only appeared later. It is possible that the insect trap was moved from its original position and / or orientation to a different position and / or orientation by wind and / or an animal and / or precipitation, where the reflections occur.

[0091] Changes in the position and / or orientation of components of the insect trap and / or the insect trap as a whole: It is conceivable that the position and / or orientation and / or orientation of components of the insect trap and / or the insect trap as a whole may change over time. Such changes can be the result of weather influences (e.g., precipitation, wind, sunlight), interactions with animals and / or people, and / or earth tremors (e.g., earthquakes, falling trees, vehicles passing by).

[0092] If an insect trap filled with a liquid was originally set up with the flat bottom of the container containing the liquid horizontal, i.e., perpendicular to the direction of gravity, so that the surface of the liquid is at the same distance from the bottom of the container at all points, it is possible that the orientation has changed and the liquid surface is no longer at the same distance from the bottom at all points. This can lead to distortions in the images.

[0093] It is also possible that the insects accumulate in one or more places in the collection container.

[0094] Furthermore, it is possible that the alignment of the camera and / or optical elements of the camera with respect to the collection area has changed and that the collection area is no longer or no longer completely imaged and / or is imaged completely or partially out of focus.

[0095] Camera is defective: It is possible that the camera is defective and the resulting images are not suitable for automated detection, counting, and / or identification of insects in the collection area. For example, the resulting images may be noisy, have a color cast, have a low contrast range, and / or be completely black or white.

[0096] Camera does not produce images of the collection area: It is possible that the camera produces images during maintenance of the insect trap and / or during assembly of the insect trap that do not show the collection area of ​​the insect trap, but rather, for example, other components of the insect trap and / or the surrounding area of ​​the insect trap. Such images may be unsuitable for automated detection, counting and / or identification of insects in the collection area of ​​the insect trap. Such images can also be identified using the means described in this description and, for example, sorted out. Sorting out may mean that a sorted image is not automatically analyzed to detect, count and / or identify insects in the collection area.

[0097] The functional impairments and / or their effects are captured in images generated by the camera in the camera-monitored insect trap. The images are used to train a machine learning model to automatically detect such functional impairments. The term "automatic" means without human intervention.

[0098] Such a "machine learning model" can be understood as a computer-implemented data processing architecture. The model can receive input data and produce output data based on this input data and model parameters. Through training, the model can learn a relationship between the input data and the output data. During training, model parameters can be adjusted to produce a desired output for a given input.

[0099] When training such a model, the model is presented with training data from which it can learn. The trained machine learning model is the result of the training process. The training data includes input data and the correct output data (target data) that the model is supposed to generate based on the input data. During training, patterns are recognized that map the input data to the target data.

[0100] During the training process, the input data of the training data is fed into the model, and the model generates output data. The output data is compared with the target data. Model parameters are modified so that the deviations between the output data and the target data are reduced to a (defined) minimum. An optimization method such as a gradient descent method can be used to modify the model parameters to reduce the deviations.

[0101] The deviations can be quantified using a loss function. Such an error function can be used to calculate an error (loss) for a given pair of output and target data. The goal of the training process can be to modify (adjust) the parameters of the machine learning model so that the error is reduced to a (defined) minimum for all pairs in the training dataset.

[0102] For example, if the output and target data are numbers, the error function can be the absolute difference between these numbers. In this case, a high absolute error may indicate that one or more model parameters need to be significantly changed.

[0103] For output data in the form of vectors, for example, difference metrics between vectors such as the mean square error, a cosine distance, a norm of the difference vector such as a Euclidean distance, a Chebyshev distance, an Lp norm of a difference vector, a weighted norm, or another type of difference metric of two vectors can be chosen as the error function.

[0104] For higher-dimensional outputs, such as two-dimensional, three-dimensional, or higher-dimensional outputs, an element-wise difference metric can be used. Alternatively or additionally, the output data can be transformed, e.g., into a one-dimensional vector, before calculating an error value.

[0105] In this case, the machine learning model receives one or more image recordings as input data. The model can be trained to output information for each image recording, indicating whether the image recording(s) are one or more images of a malfunctioning insect trap or one or more images of a non-malfunctioning insect trap. In other words, the machine learning model can be trained to distinguish between images of malfunctioning insect traps and images of non-malfunctioning insect traps.

[0106] The machine learning model can be trained to assign the one or more image recordings to one of at least two classes, wherein at least a first class represents image recordings of insect traps that do not have functional impairments and at least a second class represents image recordings of insect traps that have a functional impairment.

[0107] The machine learning model is trained based on training data. The training data comprises a large number of images of one or more insect traps. The term “large number” means more than 10, preferably more than 100. The images serve as input data. Some of the images may show the one or more insect traps without any functional impairments, i.e., in a state in which they are functioning properly. Another part of the images may show the one or more insect traps with a functional impairment. In addition to the input data, the training data may also include target data. The target data may indicate for each image whether the insect trap depicted in the image has a functional impairment or whether it does not have any functional impairment.The target data may also include information about the type of functional impairment present in the individual case and / or how severe it is and / or what degree of severity it has.

[0108] The machine learning model can be trained to assign each image to exactly one of two classes, where exactly one class represents images of insect traps that exhibit no functional impairments, and the other class represents images of insect traps that exhibit one or more functional impairments. In other words, the machine learning model can be trained to perform binary classification. For such a case, it is sufficient to have information for each of the individual images in the training data as to whether the image shows an insect trap with a functional impairment or whether the image shows an insect trap without a functional impairment.In such a case, the machine learning model can be trained to detect insect traps with one (or more) functional impairments, regardless of the specific impairment(s). It is also possible to train a machine learning model to perform feature extraction for each image input to the machine learning model and generate a compressed representation of the image. The machine learning model can be trained to generate similar compressed representations for images that do not show an insect trap with a functional impairment.If the trained machine learning model is fed an image of an insect trap with a functional impairment, the trained machine learning model generates a compressed representation of the image of the insect trap with the functional impairment that can be distinguished from the compressed representations of the images of insect traps without functional impairment. Such training, in which the machine learning model is only trained to recognize whether a functional impairment or no functional impairment is present, can be useful if a user is only interested in knowing whether the insect trap is functioning properly or whether intervention is required to eliminate a functional impairment (whatever it may be).

[0109] The machine learning model can also be trained to recognize a specific functional impairment. The specific functional impairment can be one of the functional impairments described earlier in this description. It is possible that a user is only interested in knowing whether the specific functional impairment is present (e.g., the liquid level is too low). The machine learning model can be trained to assign each image to one of two classes, where one class represents images of insect traps that exhibit the specific functional impairment and the other class represents images of insect traps that do not exhibit the specific functional impairment, i.e., where either no functional impairment is present or a functional impairment other than the specific impairment is present.In such a case, the training data for each image capture includes information about whether the specific functional impairment is present or not in the imaged insect trap.

[0110] The machine learning model can also be trained to assign each image to one of more than two classes, where the classes represent, for example, the severity of the specific functional impairment. A first class can, for example, represent images of insect traps where the specific functional impairment does not occur (e.g., no contamination); a second class can represent images of insect traps where the specific functional impairment is slightly present (e.g., slight contamination); a third class can represent images of insect traps where the specific functional impairment is clearly present (e.g., significant contamination). A slight functional impairment can mean that the insect trap is still sufficiently functional, but that maintenance is necessary in the future to avoid further functional impairment.A significant or severe impairment may indicate that immediate maintenance is required. More than the three levels mentioned are also conceivable, e.g., four (e.g., no impairment, slight impairment, moderate impairment, severe impairment) or more and / or other levels. In such a case, the training data for each image capture includes information about whether the impairment is present in the depicted insect trap, and if so, how severe it is and / or with what severity.

[0111] The machine learning model can also be trained to recognize more than one specific functional impairment, i.e., to distinguish between different functional impairments. For example, the machine learning model can be trained to learn a number n of specific functional impairments, where n is an integer greater than 1. The machine learning model can be trained to assign each image to one of at least n+1 classes, where a first class represents images of insect traps that do not exhibit any functional impairment, and each of the at least n remaining classes represents images of insect traps that exhibit one of the n specific functional impairments. It is also possible for the machine learning model to be additionally trained to recognize two or more severity levels of one or more of the n specific functional impairments.In other words, the machine learning model can be trained to recognize the severity and / or level of severity of one or more of the n specific functional impairments. In such a case, the training data includes, for each image acquisition, information about whether a functional impairment is present, if a functional impairment is present, which specific functional impairment is present, and, for one or more of the specific functional impairments, the severity and / or level of severity of the impairment.

[0112] Existing camera-monitored insect traps can be used to generate the training data described in this description. Camera-monitored insect traps can be operated for a period of time. The images generated by the camera-monitored insect traps can be analyzed by one or more experts. The one or more experts can annotate each image with one of the information required for training (annotations), which is then used as target data. The one or more experts can review the images and annotate each image with information indicating whether the respective image shows an insect trap with or without functional impairment.If necessary for training the machine learning model, each image showing an insect trap with a functional impairment can be annotated with information about the severity of the impairment and / or its occurrence. If necessary for training the machine learning model, each image showing an insect trap with a functional impairment can be annotated with information about the specific functional impairment present.

[0113] When training the machine learning model, the images are fed (one after the other) to the machine learning model. The machine learning model can be configured to assign each image to one of at least two classes. The class assignment can be output by the machine learning model, for example, in the form of a number. For example, the number 0 can represent images of insect traps that show no functional impairment; the number 1 can represent images of insect traps that exhibit a first specific functional impairment; the number 2 can represent images of insect traps that exhibit a second specific functional impairment, etc.

[0114] It is also possible for the machine learning model to be configured to output a vector for each image, wherein the vector includes a number for each functional impairment at a coordinate of the vector, which number indicates whether the respective functional impairment is shown in the image (i.e., present in the depicted insect trap) or not shown (i.e., not present in the depicted insect trap). Such an approach has the advantage that different functional impairments that are present simultaneously in an insect trap can also be detected alongside one another. In such a vector, the number 0 can indicate that a specific functional impairment is not present, and the number 1 can indicate that the specific functional impairment is present.The position in the vector (coordinate) at which the respective number occurs can provide information about which specific functional impairment is involved.

[0115] It is also possible for the machine learning model to be configured to specify a probability for one or more (specific) functional impairments that the (specific) functional impairment will occur in the respective insect trap depicted. The probability can, for example, be specified as a value in the range from 0 to 1, with the higher the value, the higher the probability.

[0116] It is also possible that the machine learning model is configured to output a severity level for one or more (specific) functional impairments with which the (specific) functional impairment occurs in the respective insect trap depicted.

[0117] The output (output data) produced by the machine learning model based on an input image can be compared with the target data. Using an error function, deviations between the output data and the target data can be quantified. In an optimization procedure (e.g., a gradient descent method), the deviations can be reduced by modifying model parameters. If the deviations reach a (predefined) minimum or a plateau, training can be terminated. The trained machine learning model can be used to detect one or more functional impairments and, optionally, their severity in an insect trap.

[0118] For this purpose, a new image of a collection area of ​​an insect trap can be fed into the machine learning model. The term "new" means that the corresponding image has not already been used to train the machine learning model. The trained machine learning model assigns the new image to one of at least two classes that were used to train the machine learning model. The trained machine learning model outputs information about the class to which the machine learning model has assigned the image. It is possible that the trained machine learning model outputs information about the probability that one or more functional impairments are present and / or their severity and / or their degree of severity.

[0119] The output of the machine learning model may be displayed on a screen, printed on a printer, stored in a data store, and / or transmitted to a separate computer system (e.g., over a network).

[0120] The output of the machine learning model can be used to automatically sort out images of insect traps with one (or multiple) functional impairments. In such a case, an image of an insect trap is analyzed in a first step according to the present disclosure for the presence of one (or multiple) functional impairments before being analyzed in a subsequent second step to detect, count, and / or identify insects in the collection area of ​​the insect trap depicted in the image. It is possible that only those images for which the analysis in the first step has shown that they do not have a functional impairment are fed to the second step for detecting, counting, and / or identifying insects.It is also possible that only those images for which the first analysis revealed no functional impairment or only a functional impairment with a low degree of severity are submitted to the second analysis for the detection, counting, and / or identification of insects. Likewise, it is possible that only images with specific functional impairments or with specific functional impairments of a predefined degree of severity, or with a minimum number of different functional impairments, are sorted out. It is possible for a user to specify in advance which images should be sorted out.

[0121] If the trained machine learning model assigns an image to a class that represents images of insect traps with a functional impairment, a message can be issued to a user. Such a message can inform the user that a functional impairment of an insect trap exists. The message can inform the user that an image will not be analyzed to detect, count, and / or identify insects because the insect trap that is at least partially depicted in the image has a functional impairment. A message to a user can include the following information: which insect trap is affected (it can, for example,a location of the insect trap must be specified), what functional impairment exists, how severe is the functional impairment, what measures can be taken to restore the full functionality of the insect trap, and when should the measures be taken to prevent further functional impairment). It is also possible that the image recording in which the trained machine learning model detected a functional impairment is also displayed to the user so that the user can form their own opinion about the functional impairment.

[0122] If the output of the trained machine learning model is a probability value for the presence of a functional impairment, this probability value can be compared to a predefined threshold. If the probability value is greater than or equal to the threshold, a notification can be issued to a user regarding the presence of a functional impairment in an insect trap. If the probability value is less than the threshold, the image can be analyzed to detect, count, and / or identify insects in the collection area of ​​the insect trap.

[0123] It is conceivable that there is more than one threshold with which the probability value is compared. For example, it is possible that there is an upper threshold and a lower threshold. If the probability value is below the lower threshold, the probability of a functional impairment is so low that the user does not need to be informed. The image can be submitted to an analysis to detect, count and / or identify insects in the collection area of ​​the insect trap. If the probability value is above the upper threshold, the probability of a functional impairment is so high that a notification about the functional impairment is issued to the user. It is possible that the image is not submitted to an analysis to detect, count and / or identify insects.If the probability value lies in the range from the lower threshold to the upper threshold, there is a certain degree of uncertainty as to whether or not a functional impairment exists. This uncertainty may result from the image being of comparatively low quality. It is possible that a command is sent to the insect trap's control unit to generate another image, so that this additional image can also be fed into the trained machine learning model for detecting a functional impairment. It is possible that parameters are changed when generating the additional image in order to improve the quality of the image. For example, the exposure time can be increased and / or the illumination of the collection area can be increased by one or more lighting units and / or filters (color filters, polarization filters, and / or the like) can be used.Further image acquisition can then provide clarity as to whether or not a functional impairment exists. However, it is also possible that the uncertainty regarding the presence of a functional impairment results from the fact that a functional impairment is only just becoming apparent, i.e., that only a relatively minor functional impairment is present (e.g., slight contamination). It is possible that the insect trap's control unit is instructed by a command to reduce the time interval between two consecutive image acquisitions. Images are then acquired at a shorter time interval to detect any further functional impairment and / or an increase in the severity of the functional impairment at an early stage.

[0124] Thresholds can be set by an expert based on their experience. They can also be set by a user. The user can decide for themselves whether they want to be informed when the probability of a functional impairment is low, or whether they would prefer to be informed when the probability of a functional impairment is comparatively high.

[0125] The machine learning model of the present disclosure may be or include an artificial neural network. An "artificial neural network" includes at least three layers of processing elements: a first layer with input neurons (nodes), a kth layer with at least one output neuron (node), and k-2 inner layers, where k is a natural number greater than 2.

[0126] The input neurons are used to receive the input representations. Typically, there is one input neuron for each pixel of an image input to the artificial neural network. Additional input neurons may be present for additional input values ​​(e.g., information about the image capture, the imaged insect trap, camera parameters, weather conditions during image capture, and / or the like).

[0127] The output neurons can be used to output information about which class the input image was assigned to and / or with which probability it was assigned to the class.

[0128] The processing elements of the layers between the input neurons and the output neurons are connected in a predetermined pattern with predetermined connection weights.

[0129] The neural network can be trained, for example, using a backpropagation method. The goal is to achieve the most reliable mapping of the input data to the target data for the network. The quality of the prediction is described by an error function. The goal is to minimize the error function. In the backpropagation method, an artificial neural network is trained by changing the connection weights.

[0130] In the trained state, the connection weights between the processing elements contain information regarding the relationship between image recordings and functional impairments of the insect traps depicted in the image recordings. This information can be used to detect functional impairments of an insect trap early on based on a new image recording. The term "new" means that the new image recording was not already used in training the artificial neural network.

[0131] A cross-validation method can be used to split the data into training and validation sets. The training set is used for backpropagation training of the network weights. The validation set is used to test the predictive accuracy of the trained network when applied to unknown (new) data.

[0132] The artificial neural network can be a so-called convolutional neural network (CNN for short) or it can include one.

[0133] A convolutional neural network (“CNN”) is capable of processing input data in the form of a matrix. This makes it possible to use images represented as a matrix (e.g., width x height x color channels) as input data. A neural network, e.g., in the form of a multi-layer perceptron (MLP), requires a vector as input. This means that in order to use an image as input, the image elements (pixels) of the image would have to be rolled out one after the other in a long chain. This means that multi-layer perceptrons are not able to recognize objects in an image regardless of the object's position in the image. The same object at a different position in the image would have a different input vector.

[0134] A CNN usually consists essentially of filters (convolutional layer) and aggregation layers (pooling layer), which repeat alternately, and at the end of one or more layers of fully connected neurons (dense / fully connected layer).

[0135] Numerous architectures of artificial neural networks are described in the scientific literature that are used to assign an image to a class (image classification). Examples are Xception (see e.g.: F. Chollet: EfficientNet (see e.g.: T. Mingxing Tan et al. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks, arXiv:1905.11946v5), DenseNet (see e.g.: G. Wang et al. Study on Image Classification Algorithm Based on Improved DenseNet, Journal of Physics: Conference Series, 2021, 1952, 022011), Inception (see e.g. J. Bankar et al.Convolutional Neural Network based Inception v3 Model for Animal Classification, International Journal of Advanced Research in Computer and Communication Engineering, 2018, Vol. 7, Issue 5) and others (see e.g.: M. Tripathi: Analysis of Convolutional Neural Network based Image Classification Techniques. Journal of Innovative Image Processing, 2021, 3, 100-117; K. He et al. : Deep Residual Learning for Image Recognition, arXiv:1512.03385vl; M. Aamir et al.: An Optimized Architecture of Image Classification Using Convolutional Neural Network, International Journal of Image, Graphics and Signal Processing, 2019, 11, 30-39).

[0136] The machine learning model of the present disclosure may have such or a similar architecture.

[0137] The machine learning model of the present disclosure may be or include a transformer. A transformer is a model that can translate one sequence of characters into another sequence of characters, taking into account dependencies between distant characters. Such a model can be used, for example, to translate text from one language to another. A transformer includes encoders and decoders connected in series. Transformers have already been successfully used to classify images (see, for example: A. Dosovitskiy et al.: An image is worth 16 x 16 Words: Transformers for image recognition at scale, arXiv:2010.11929v2; A. Khan et al.: Transformers in Vision: A Survey, arXiv:2101.01169v5). The machine learning model of the present disclosure may have a hybrid architecture, for example, combining elements of a CNN with elements of a transformer.

[0138] The machine learning model of the present disclosure can be initialized using standard methods (random initialization, He initialization, Xavier initialization, etc.). However, it can also be pre-trained based on publicly available, already annotated images (see, for example, https: / / www.image-net.org). The training of the machine learning model can therefore be based on initialization or pre-training and can also include transfer learning, so that only parts of the weights / parameters of the machine learning model are retrained.

[0139] The machine learning model may feature an autoencoder architecture. An "autoencoder" is an artificial neural network that can be used to learn efficient data encodings in an unsupervised learning process. Generally, the task of an autoencoder is to learn a compressed representation for a dataset and thus extract essential features. This allows it to be used for dimensionality reduction by training the network to ignore "noise." An autoencoder comprises an encoder, a decoder, and a layer between the encoder and decoder that has a lower dimension than the encoder's input layer and the decoder's output layer.This layer (often referred to as bottleneck, encoding, or embedding) forces the encoder to generate a compressed representation of the input data that minimizes noise and is sufficient for the decoder to reconstruct the input data. The autoencoder can therefore be trained to generate a compressed representation of an input image. For example, the autoencoder can be trained exclusively on images of insect traps that exhibit no functional impairment. However, the autoencoder can also be trained on images that show insect traps with and without functional impairment. Once the autoencoder is trained, the decoder can be discarded, and the encoder can be used to generate a compressed representation for each input image.When a new insect trap is installed, an initial image of the insect trap can be generated after installation. This initial image, which the insect trap captures without any functional impairment, can be used as a reference. A compressed representation of the initial image can be generated using the encoder of the trained autoencoder. This compressed representation is the reference representation. During operation of the insect trap, compressed representations of images generated by the insect trap's camera can be generated using the encoder. The more similar a compressed representation is to the reference image, the less likely it is that a functional impairment is present. The more a compressed representation differs from the reference image, the more likely it is that a functional impairment is present.The similarity of representations can be quantified using a similarity or distance measure. Examples of such similarity or distance measures are cosine similarity, Manhattan distance, Euclidean distance, Minkowski distance, Lp norm, and Chebyshev distance. If a distance measure exceeds or falls below a predefined threshold, which can be set by an expert or specified by a user, a notification can be issued indicating a functional impairment and / or the image capture can be discarded.

[0140] An example of an autoencoder architecture is the U-Net (see, for example, O. Ronneberger et al.-. U-net: Convolutional networks for biomedical image segmentation, International Conference on Medical image computing and computer-assisted intervention, pages 234-241, Springer, 2015, https: / / doi.org / 10.1007 / 978-3-319-24574-4_28).

[0141] It is also possible to use an architecture for the machine learning model of the present invention as described, for example, in the following publication: J. Dippel, S. Vogler, S. Höhne: Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling, arXiv:2104.04323v2. The autoencoder described therein comprises, in addition to an encoder and a decoder, a strand (projection head) which, based on the compressed representation generated by the encoder, generates an output indicating whether the image recording shows an insect trap with or without a functional impairment.In other words, the autoencoder is not only trained to generate a compressed representation of the input data and reconstruct the input data based on the compressed representation, but the autoencoder is also simultaneously trained to distinguish images of insect traps with functional impairment from images of insect traps without functional impairment (contrastive reconstruction). Once the autoencoder is trained, the encoder can be used to generate a compressed reference representation for the first image of a newly installed insect trap. This compressed reference representation is compared with compressed representations of images generated during operation of the insect trap, and in the event of a defined deviation, a notification is issued indicating a functional impairment.It is also possible to use the encoder together with the projection operator (projection head) directly for classification.

[0142] Further techniques for classifying images are described, for example, in SVS Prasad et al.: Techniques in Image Classification - A Survey, Global Journal of Researches in Engineering (F), 2015, Volume XV, Issue VI, Version I, 17-32; K. Sanghvi et al. '. A Survey on Image Classification Techniques, 2020, http: / / dx.doi.org / 10.2139 / ssrn.3754116 and can also be used to implement the present invention.

[0143] The invention is explained in more detail below with reference to drawings, without wishing to limit the invention to the features and combinations of features shown in the drawings.

[0144] Fig. 1 shows a schematic and exemplary training process for a machine learning model. The machine learning model is trained using training data TD. The training data TD comprises a large number of images. Each image I shows a collection area of ​​an insect trap (not shown in Fig. 1). The training data TD further comprises, for each image I, information A indicating whether the insect trap depicted in the image I has a functional impairment or whether it does not have a functional impairment. It is possible that the information A includes information about which specific functional impairment is present in the individual case and / or how severe the functional impairment is and / or what degree of severity the functional impairment has. In Fig.1, for the sake of clarity, only one training data set comprising an image I with information A is shown; however, the training data comprises a large number of such training data sets. The image I represents input data for the MLM machine learning model. The information A represents target data for the MLM machine learning model. The image I is fed to the MLM machine learning model. The MLM machine learning model assigns the image to one of at least two classes. The assignment is made based on the image I and on the model parameters MP. The MLM machine learning model outputs information O that indicates the class the image has been assigned to and / or the probability that the image has been assigned to one or more of the at least two classes. The output information O is compared with the information A.An error function LF is used to quantify the deviations between the information O (output) and the information A (target data). For each pair of information A and information O, an error value LV can be calculated. The error value LV can be reduced in an optimization procedure (e.g., a gradient method) by modifying model parameters MP. The goal of training can be to reduce the error value for all image acquisitions to a predefined minimum. Once the predefined minimum is reached, training can be terminated.

[0145] Fig. 2 schematically shows the use of a trained machine learning model to detect a functional impairment in an insect trap. The trained model MLM 1 of machine learning may have been trained in a training procedure as described in relation to Fig. 1. The trained model MLM 1A new image I* is fed into the machine learning process. The new image I* shows a collection area of ​​an insect trap. The model assigns the new image I* to one of at least two classes for which the trained model MLM 1 of machine learning. The trained model MLM 1 The machine learning algorithm outputs information O that indicates which class the image was assigned to and / or the probability that the image was assigned to one or more of the at least two classes. The information O can be output to a user.

[0146] Fig. 3 schematically shows another example of training a machine learning model. The machine learning model has an autoencoder architecture. The autoencoder AE comprises an encoder E and a decoder D. The encoder E is configured to generate a compressed representation CR for an image recording I based on model parameters MP. The decoder D is configured to generate a reconstructed image recording RI based on the compressed representation CR and model parameters MP, which is as close as possible to the image recording I. An error function LF can be used to quantify deviations between the image recording I and the reconstructed image recording RI. The deviations can be minimized in an optimization process (e.g., in a gradient method) by modifying model parameters MP.The autoencoder AE is typically trained using an unsupervised learning process based on a large number of image recordings. Only one image recording I of the large number of image recordings is shown in Fig. 3. Each image recording of the large number of image recordings shows a collection area or a portion thereof of one or more insect traps. The one or more insect traps may have one or more functional impairments or be free of functional impairments.

[0147] Fig. 4 schematically shows another example of using a trained machine learning model to detect a functional impairment in an insect trap. The trained machine learning model may have been trained using a training process as described with reference to Fig. 3. The trained machine learning model may be an encoder E of an autoencoder. The encoder E is shown twice in Fig. 4; however, it is the same encoder in both cases; it is shown twice merely to illustrate the detection process. In a first step, a first image recording L* of a collection area of ​​an insect trap is fed to the encoder E. The asterisk * indicates that the image recording L* was not used to train the machine learning model.The image recording L* is preferably an image recording of an insect trap without any functional impairment, which can, for example, have been created after the insect trap has been installed. The encoder E is configured to generate a first compressed representation CRi for the first image recording L*. The first compressed representation CRi can be used as a reference representation. It can be stored in a data memory. During operation of the insect trap, further images of the collection area of ​​the insect trap are created. Fig. 4 shows one of these further images, the image recording L*. The image recording L* is also fed to the encoder E. The encoder E generates a second compressed representation CR2 for the image recording L*. The first representation CRi and the second representation CR2 are compared with each other in a next step.During this comparison, a distance measure D is calculated that quantifies the differences between the first representation CRi and the second representation CR2. In a next step, the distance measure is compared with a predefined threshold value T. If the distance measure is greater than the predefined threshold value T ("y"), a message M is issued. The message M includes information that the insect trap shown in the image recording I2* has a functional impairment. If the distance measure is not greater than the predefined threshold value T ("n"), the image recording I2* is fed into an analysis DCI(l2*) in order to detect insects in the collection area of ​​the insect trap and / or to count and / or identify the insects located in the collection area. Fig. 5 shows an example and schematically a computer-implemented method for training a machine learning model in the form of a flowchart.

[0148] The training procedure (100) includes the following steps:

[0149] (110) Receiving and / or providing training data, wherein the training data comprise input data and target data, o wherein the input data comprise a plurality of image recordings of one or more insect traps, wherein each image recording shows at least part of a collection area of ​​an insect trap, o wherein the target data comprise a class assignment for each image recording, wherein the class assignment indicates which class of at least two classes the image recording is assigned to, wherein at least a first class represents image recordings of insect traps that do not have a functional impairment, and at least a second class represents image recordings of insect traps that have a functional impairment,

[0150] (120) Providing a machine learning model, wherein the machine learning model is configured to assign the image recording to one of the at least two classes based on an image recording and on the basis of model parameters,

[0151] (130) Training the machine learning model with the training data, wherein the training comprises for each image acquisition:

[0152] (131) Feeding the image capture to the machine learning model,

[0153] (132) receiving an output from the machine learning model, the output indicating which class the machine learning model has assigned the image recording to and / or with what probability the machine learning model has assigned the image recording to one or more of the at least two classes,

[0154] (133) Determining a deviation between the output and the class assignment,

[0155] (134) Minimizing the deviation by modifying the model parameters,

[0156] (140) storing and / or outputting the trained machine learning model and / or using the trained machine learning model to detect a malfunction in a camera-monitored insect trap.

[0157] Fig. 6 shows an example and schematically a computer-implemented method for detecting a functional impairment in a camera-monitored insect trap.

[0158] The detection process (200) comprises the following steps:

[0159] (210) receiving an image recording, the image recording showing a collection area of ​​an insect trap,

[0160] (220) Providing a trained machine learning model, wherein the machine learning model is configured and trained on the basis of training data to assign image recordings to one of at least two classes, wherein the training data comprises input data and target data,

[0161] • wherein the input data comprises a plurality of images of one or more insect traps, each image showing at least part of a collection area of ​​an insect trap, • wherein the target data for each image comprises a class assignment, the class assignment indicating which of at least two classes the image is assigned to, at least a first class representing images of insect traps that do not have a functional impairment and at least a second class representing images of insect traps that do have a functional impairment,

[0162] (230) Feeding the image capture to a trained machine learning model,

[0163] (240) Receiving information from the machine learning model as to which class the image recording was assigned,

[0164] (250) in the event that the image recording has been assigned to one of the at least one second class: issuing a message that the insect trap shown at least partially in the image recording has a functional impairment.

[0165] The steps, methods and / or functions described in this disclosure may be performed in whole or in part by a computer system.

[0166] A "computer system" is an electronic data processing system that processes data using programmable computing instructions. Such a system typically includes a "computer," the unit that includes a processor for performing logical operations, and peripherals.

[0167] In computer technology, "peripherals" refers to all devices connected to a computer that serve to control the computer and / or act as input and output devices. Examples include monitors, printers, scanners, mice, keyboards, drives, cameras, microphones, speakers, etc. Internal connectors and expansion cards are also considered peripherals in computer technology.

[0168] Today's computer systems are often divided into desktop PCs, portable PCs, laptops, notebooks, netbooks, and tablet PCs, as well as so-called handheld devices (e.g., smartphones); all of these systems can be used to implement the invention.

[0169] The term "computer" should be broadly interpreted to include any type of electronic device with data processing capabilities, including, as non-limiting examples, personal computers, servers, embedded cores, communications devices, processors (e.g., digital signal processors (DSPs), microcontrollers, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc.) and other electronic computing devices.

[0170] The term "process," as used above, is intended to encompass any type of calculation, manipulation, or transformation of data that is represented as physical, e.g., electronic, phenomena and that may occur or be stored, e.g., in registers and / or memory of at least one computer or processor. The term "processor" includes a single processing unit or a plurality of distributed or remote processing units.

[0171] Fig. 7 shows an exemplary and schematic representation of an embodiment of a computer system of the present disclosure. The computer system (1) comprises an input unit (10), a control and computing unit (20), and an output unit (30).

[0172] The control and computing unit (20) is configured to cause the input unit to receive an image recording, wherein the received image recording shows a collection area of ​​an insect trap, to feed the received image recording to a trained machine learning model, wherein the machine learning model is configured and has been trained on the basis of training data to assign image recordings to one of at least two classes, wherein the training data comprises input data and target data, o wherein the input data comprises a plurality of image recordings of one or more insect traps, wherein each image recording shows at least part of a collection area of ​​an insect trap, o wherein the target data for each image recording comprises a class assignment, wherein the class assignment indicates which class of at least two classes the image recording is assigned to, wherein at least a first class represents image recordings of insect traps,which do not exhibit any functional impairment and at least a second class represents images of insect traps which exhibit a functional impairment, to receive information from the machine learning model as to which class of the at least two classes the image recording was assigned to and / or with what probability the image recording was assigned to one or more of the at least two classes, to cause the output unit to output a message that the insect trap shown at least partially in the image recording has a functional impairment if the image recording was assigned to one of the at least one second class.

[0173] Fig. 8 shows an exemplary and schematic illustration of another embodiment of a computer system of the present disclosure.

[0174] The computer system (1) comprises a processing unit (20) connected to a memory (50).

[0175] The processing unit (20) may comprise one or more processors alone or in combination with one or more memories. The processing unit (20) may be conventional computer hardware capable of processing information such as digital images, computer programs, and / or other digital information. The processing unit (20) typically consists of an arrangement of electronic circuits, some of which may be embodied as an integrated circuit or as multiple interconnected integrated circuits (an integrated circuit is sometimes referred to as a "chip"). The processing unit (20) may be configured to execute computer programs that may be stored in a main memory of the processing unit (20) or in the memory (50) of the same or another computer system.

[0176] The memory (50) may be ordinary computer hardware capable of storing information such as digital images (e.g., representations of the examination area), data, computer programs, and / or other digital information either temporarily and / or permanently. The memory (50) may comprise volatile and / or non-volatile memory and may be permanently installed or removable. Examples of suitable memories include RAM (Random Access Memory), ROM (Read-Only Memory), a hard disk, flash memory, a removable computer diskette, an optical disc, magnetic tape, or a combination of the above. Optical discs may include read-only compact discs (CD-ROM), read / write compact discs (CD-R / W), DVDs, Blu-ray discs, and the like.

[0177] In addition to the memory (50), the processing unit (20) can also be connected to one or more interfaces (11, 12, 30, 41, 42) for displaying, transmitting, and / or receiving information. The interfaces can comprise one or more communication interfaces (41, 42) and / or one or more user interfaces (11, 12, 30). The one or more communication interfaces (41, 42) can be configured to send and / or receive information, e.g., to and / or from an MRI scanner, a CT scanner, an ultrasound camera, other computer systems, networks, data storage devices, or the like. The one or more communication interfaces (41, 42) can be configured to transmit and / or receive information via physical (wired) and / or wireless communication connections.The one or more communication interfaces (41, 42) may include one or more interfaces for connecting to a network, e.g., using technologies such as cellular, Wi-Fi, satellite, cable, DSL, fiber optic, and / or the like. In some examples, the one or more communication interfaces (41, 42) may include one or more short-range communication interfaces configured to connect devices using short-range communication technologies such as NFC, RFID, Bluetooth, Bluetooth LE, ZigBee, infrared (e.g., IrDA), or the like.

[0178] The user interfaces (11, 12, 30) may comprise a display (30). A display (30) may be configured to display information to a user. Suitable examples include a liquid crystal display (LCD), a light-emitting diode display (LED), a plasma display (PDP), or the like. The user input interface(s) (11, 12) may be wired or wireless and may be configured to receive information from a user into the computer system (1), e.g., for processing, storage, and / or display. Suitable examples of user input interfaces (11, 12) include a microphone, an image or video capture device (e.g., a camera), a keyboard or keypad, a joystick, a touch-sensitive surface (separate from or integrated with a touchscreen), or the like.In some examples, the user interfaces may include automatic identification and data capture (AIDC) technology for machine-readable information. This may include barcodes, radio frequency identification (RFID), magnetic stripes, optical character recognition (OCR), integrated circuit cards (ICC), and the like. The user interfaces may further include one or more interfaces for communicating with peripheral devices such as printers and the like.

[0179] One or more computer programs (60) may be stored in memory (50) and executed by the processing unit (20), which is thereby programmed to perform the functions described in this description. The retrieval, loading, and execution of instructions of the computer program (60) may occur sequentially, such that one instruction is retrieved, loaded, and executed at a time. However, the retrieval, loading, and / or execution may also occur in parallel.

Claims

Patent claims 1. Computer-implemented method comprising: Receiving an image recording, wherein the received image recording shows at least part of a collection area of ​​an insect trap, Providing a trained machine learning model, wherein the machine learning model is configured and trained on the basis of training data to distinguish images of insect traps with a functional impairment from images of insect traps without a functional impairment, Feeding the received image to the trained machine learning model, Receiving information from the machine learning model, the information indicating whether the image recording shows an insect trap with a functional impairment, in the event that the received image recording shows an insect trap with a functional impairment: issuing a notification that the insect trap shown at least partially in the received image recording has a functional impairment.

2. The method according to claim 1, wherein the machine learning model is configured and trained on the basis of training data to assign image recordings to one of at least two classes, wherein the training data comprises input data and target data, o wherein the input data comprises a plurality of image recordings of one or more insect traps, wherein each image recording shows at least part of a collection area of ​​an insect trap, o wherein the target data for each image recording comprises a class assignment, wherein the class assignment indicates which class of at least two classes the image recording is assigned to, wherein at least a first class represents image recordings of insect traps that do not have a functional impairment, and at least a second class represents image recordings of insect traps that have a functional impairment,wherein receiving information from the machine learning model comprises: o receiving information from the machine learning model as to which class the image recording was assigned, wherein the notification that the insect trap shown at least partially in the received image recording has a functional impairment is output if the received image recording was assigned to one of the at least one second class.

3. The method of claim 2, wherein training the machine learning model for each image capture of the plurality of image captures comprises: Inputting the image into the machine learning model, wherein the machine learning model is configured to assign the image to one of at least two classes based on the input image and model parameters, Receiving an output from the machine learning model, the output indicating which of the at least two classes the input image recording was assigned to by the machine learning model, Quantifying a deviation between the output and the class assignment of the training data, Minimize the deviation by modifying model parameters.

4. The method according to any one of claims 1 to 3, wherein the machine learning model is configured and trained to assign the received image to one of a plurality of classes, each class of a plurality of classes having a specific functional impairment, wherein outputting the notification comprises: outputting a notification indicating which specific functional impairment is present in the insect trap shown at least partially in the received image.

5. The method according to any one of claims 1 to 4, wherein the machine learning model is configured and trained to output, based on the received image recording, for one or more specific functional impairments, a probability with which the specific functional impairment occurs, wherein outputting the message comprises: outputting a message with which probability the one or more specific functional impairments are present in the insect trap shown at least partially in the received image recording.

6. The method according to any one of claims 1 to 4, wherein the machine learning model is configured and trained to output a severity level for one or more specific functional impairments based on the received image recording, wherein outputting the message comprises: outputting a message indicating the severity level of the one or more specific functional impairments in the insect trap shown at least partially in the received image recording.

7. The method according to any one of claims 1 to 6, wherein the functional impairment and / or the one or more specific functional impairments are selected from the following list: Liquid level in the collection area has fallen below a lower threshold, Liquid level in the collection area has risen above an upper threshold, Collection area is dirty, Collection area is exhausted, Algae formation in the collection area, Foam in the collection area, Ice formation or icing of liquids, The camera’s field of view is limited, Camera and / or optical elements are dirty, Lighting source(s) defective and / or dirty, - TI - unwanted reflexes occur, Position and / or location of components of the insect trap and / or the insect trap as a whole are changed, Camera is defective, Camera does not capture images of the collection area.

8. The method according to claim 1, wherein the machine learning model is configured and trained to generate a compressed representation for the received image recording, the method further comprising: o quantifying a similarity and / or a difference between the compressed representation and a reference representation by calculating a similarity measure and / or a distance measure, wherein outputting the notification comprises: outputting the notification that the insect trap shown at least partially in the received image recording has a functional impairment if the similarity measure is below a predefined threshold and / or the distance measure is above a predefined threshold.

9. The method according to any one of claims 1 to 8, wherein the machine learning model is or comprises an artificial neural network, wherein the artificial neural network is preferably a CNN or comprises such and / or is or comprises such a transformer.

10. The method according to any one of claims 1 to 8, wherein the machine learning model is or comprises an encoder of an autoencoder.

11. The method according to any one of claims 1 to 9, wherein the method further comprises: exclusively in the event that the received image shows an insect trap without functional impairment: detecting, counting and / or identifying insects in the collection area of ​​the insect trap based on the received image.

12. The method according to any one of claims 1 to 11, wherein the notification further comprises one or more of the following information: Location of the insect trap, Information about the functional impairment, Information about the severity of the impairment, Information on what measures can be taken to restore the full functionality of the insect trap, Information on when action should be taken to prevent further impairment of function, the received image recording.

13. A computer system comprising: an input unit, a control and computing unit, and an output unit, wherein the control and computing unit is configured to cause the input unit to receive an image recording, wherein the received image recording shows at least part of a collection area of ​​an insect trap, to feed the received image recording to a machine learning model, wherein the machine learning model is configured and has been trained on the basis of training data to distinguish image recordings of insect traps with a functional impairment from image recordings of insect traps without a functional impairment, to receive information from the machine learning model, wherein the information indicates whether the received image recording shows an insect trap with a functional impairment, to cause the output unit to output a message, wherein the message indicatesthat the insect trap shown at least partially in the received image has a functional impairment if the information output by the machine learning model indicates that the received image shows an insect trap with a functional impairment.

14. A non-transitory computer-readable storage medium having stored thereon software instructions which, when executed by a processor of a computer system, cause the computer system to carry out the method according to any one of claims 1 to 12.

15. System comprising: • an insect trap, • a camera, • a control unit, • an analysis unit, • a transmitting unit, and • an output unit, wherein the control unit is configured to cause the camera to generate one or more images of a collection area of ​​the insect trap, wherein the analysis unit is configured to feed the one or more generated images to a machine learning model, wherein the machine learning model is configured and has been trained on the basis of training data to distinguish images of insect traps with a functional impairment from images of insect traps without a functional impairment, wherein the analysis unit is configured to receive information from the machine learning model, wherein the information indicates whether the one or more images show an insect trap with a functional impairment, wherein the transmitting unit is configured to transmit the information and / or the one or more image recordings to a separate computer system, wherein the output unit is configured to output the information.

16. Kit comprising an insect trap and a computer program product, wherein the insect trap comprises a Camera or means for recording a camera, wherein the computer program product comprises program instructions, wherein the program instructions can be loaded into a working memory of a computer system and cause the computer system to carry out the method according to one of claims 1 to 12.

Images