Sound-based insect identification system

Abstract

A sound-based insect identification system includes a sound receiving unit, a processing unit, and a database. The sound receiving unit receives a sound signal. The processing unit is coupled to the sound receiving unit and obtains sound data based on the sound signal. The database stores multiple sets of data, including multiple insect classification information reflecting known insect species and multiple wing vibration sound information corresponding to the insect classification information. The sound data is converted into transformed data containing acoustic properties. The acoustic properties in the transformed data are defined with one or more salient features. These salient features are compared with the wing vibration sound information. If the salient feature matches at least one wing vibration sound information, the sound data is associated with the insect classification information.

Description

Technical Field

[0001] This invention relates to a sound-based insect identification system, and more particularly to an insect identification system based on the sound of insect wing vibrations. Prior Technology

[0002] Insect identification technology can be applied in agriculture and pest management, public health and disease control, ecological monitoring, and scientific research. Traditional insect identification methods rely on manually collecting insect-sticky paper and then identifying the insects, which is not only inefficient and labor-intensive but also prone to misidentification. To improve the problems of manual identification, image-based visual identification technology has been developed, which obtains classification results by establishing classification identification models and comparing the appearance characteristics of insects. Recently, technologies for identifying insect species by detecting insect sounds have also been developed, such as Chinese invention patent publications CN115910077A and CN101976564A.

[0003] However, whether insects produce chirping sounds is closely related to their living environment and communication methods. Most insects communicate through chemical or visual means, such as using pheromones, colors, and movements for courtship. Therefore, only a limited number of insect species produce distinct chirping sounds that can be detected and analyzed using conventional techniques. Most common insects, such as butterflies and moths (Lepidoptera), dragonflies and damselflies (Odonata), and beetles (Coleoptera), have not evolved vocal organs, making it impossible to identify insect species and characteristics by recognizing their chirping sounds.

[0004] Therefore, those skilled in the art urgently seek to solve the problem that conventional techniques for identifying insect "chirping sounds" through hearing cannot detect insects without sound-producing organs. Summary of the Invention

[0005] The main objective of this invention is to solve the problem that conventional insect identification systems cannot identify insect characteristics and species because they cannot detect insect chirping sounds.

[0006] To address the aforementioned problems, this invention discloses a sound-based insect identification system, comprising a sound receiving unit, a processing unit, and a database. The sound receiving unit receives a sound signal. The processing unit, coupled to the sound receiving unit, obtains sound data based on the sound signal. The database stores multiple sets of data, including multiple insect classification information reflecting known insect species and multiple wing vibration sound information corresponding to the insect classification information. The sound data is converted into transformed data containing acoustic properties. These acoustic properties are defined with one or more salient features. These salient features are compared with the multiple wing vibration sound information in the data. If the salient feature matches at least one wing vibration sound information, the sound data is associated with the insect classification information corresponding to the matching wing vibration sound information.

[0007] To address the aforementioned problems, this invention also discloses a sound-based insect identification method, comprising the following steps: receiving a sound signal in a detection area, the reception of the sound signal lasting for a sampling time; obtaining sound data based on the sound signal, the sound data reflecting the change of the sound signal over time during the sampling time; converting the sound data into transformed data containing acoustic properties, the acoustic properties in the transformed data being defined as one or more salient features; comparing the salient features with a complex set of data to obtain a comparison result, the data including multiple insect classification information reflecting known insect species and multiple wing vibration sound information corresponding to the insect classification information. Wherein, if the salient feature matches at least one of the wing vibration sound information, the comparison result associates the sound data with the insect classification information corresponding to the matched wing vibration sound information. Simple Explanation of the Diagram

[0008] Figure 1 is a schematic diagram of a system block diagram according to an embodiment of the present invention. Figure 2 is a schematic diagram of a database according to an embodiment of the present invention. Figure 3 is a schematic diagram of a system block according to another embodiment of the present invention. Figure 4 shows the frequency distribution under different amplitudes. Figure 5A is a schematic diagram of the time-domain waveform of a sound signal in one embodiment of the present invention. Figure 5B is the spectrum diagram of Figure 5A. Figure 6A is a schematic diagram of the time-domain waveform of a portion of Figure 5A. Figure 6B is the spectrum diagram of Figure 6A. Figure 7 is a schematic diagram of an embodiment of the present invention. Implementation

[0009] Throughout this document, the terminology used in the description of various embodiments is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, or is not intentionally limiting the number of elements, the singular forms “a,” “an,” and “the” used herein also include the plural forms. On the other hand, the terms “comprising” and “including” are intended to be included, meaning that additional elements may be present besides those listed; when an element is described as “connected” or “coupled” to another element, that element may be directly connected or coupled to the other element or through an intermediate element; furthermore, the order in which the embodiments are described should not be construed as implying that operations or steps must adhere to the literal order, and alternative embodiments may perform steps, operations, methods, etc., in an order different from that described herein.

[0010] This invention discloses a sound-based insect identification system and method, particularly a system and method for identifying insects based on the sound of wing vibrations. Specifically, in one example, the insect identification based on this system or method is based on the sound emitted by the target insect, without involving images of the target insect (such as photo comparison). More specifically, the insect identification based on this system or method is based on the sound of the target insect's wing vibrations, which is not the insect's chirping or any other sound not caused by wing vibrations. In other words, in one example, insect chirping is excluded from identification. However, it should be understood that in the sound acquisition or sampling, the system or method receives all or most of the sound from the target insect, but only in the identification process is the sound of the target insect's wing vibrations considered. The target insects include, but are not limited to, insects in the class Insecta with membranous wings, piliferous wings, lepidotic wings, fringed wings, hemitegmen, tegmen, hemielytron, elytron, and halide wings, such as dragonflies, bees, butterflies, moths, and ladybugs.

[0011] Referring to Figure 1, this invention discloses an insect identification system 10, which includes a sound receiving unit 11, a processing unit 12, and a database 13. The sound receiving unit 11 is a sound-receiving element used to receive sound signals from its surroundings. The sound receiving unit 11 can be one or more microphones, such as one or more directional microphones. The processing unit 12 can be an electronic device with data processing capabilities, such as a computer device with a central processing unit or a handheld electronic device. The processing unit 12 generates sound data based on the sound signal. Here, "based on the sound signal" means that the sound data is associated with the sound signal. The sound data may be directly converted from the sound signal, or it may be obtained by performing one or more processing and / or analysis processes on the sound signal. The database 13 can be a cloud database, a local database stored on the electronic device, or a remote database.

[0012] The insect identification system 10 identifies and analyzes the sound based on the vibration sounds of one or more wings of a target insect. It should be understood that the sound signal received by the sound receiving unit 11 may, depending on the different sound receiving environment, include only the vibration sound of the target insect's wings, but may also include the vibration sound of the wings, insect chirping, other sounds emitted by the target insect, and background noise. When the sound signal includes sounds other than the vibration sound of the wings, the sounds other than the vibration sound of the wings can be removed so that the sound signal contains only the vibration sound of the wings, in order to perform analysis and identification.

[0013] In one example, the insect identification system 10 may be an electronic device or part of an electronic device, or a combination of an electronic device and other devices, such as a smartphone, digital camera, wearable computer (e.g., smart glasses, smartwatch), personal computer, laptop, tablet computer, gaming device, etc. For example, a smartphone may constitute the insect identification system 10 on its own, or a smartphone and an external microphone may jointly constitute the insect identification system 10, or a smartphone and an external database may jointly constitute the insect identification system 10.

[0014] The sound receiving unit 11 receives the sound signal at a sampling time interval, which is a time range rather than a single point in time. In one example, the sound data reflects the change of the sound signal over time, and this sound data may be, for example, the change of sound pressure (Pa), sound pressure level (SPL / dB), amplitude (if normalized), sound energy, frequency intensity (Hz), digital audio sample value (PCM), voltage, etc., over time. This sound data can be converted into transformed data containing acoustic properties. These acoustic properties can be defined by one or more salient features, including but not limited to sound pressure, sound pressure level, amplitude, sound energy, frequency intensity, PCM, or voltage. In some examples, the sound data and the acoustic properties may be different types of sound-related physical quantities.

[0015] In one example, the salient feature is the acoustic property that accounts for the largest proportion in the sound data. For example, the sound data is a time-domain waveform, which is a waveform showing the change of sound pressure over time. This time-domain waveform can be converted into a spectrum, i.e., the converted data. In this spectrum, the sound frequency with the largest proportion is X Hz, and the sound frequency X is the salient feature. In one example, since the target insect may not continuously flap its wings, or the sound collection time may include too many non-flapping intervals, making it impossible to accurately determine the data during analysis, the ratio of a sound interval in the sampling time to the sampling time must not be less than a preset value, such as 2 / 3. The sound interval can be defined as the time when the detected acoustic energy (e.g., sound pressure, amplitude, frequency intensity, etc.) exceeds a threshold value. However, the present invention is not limited to this; the preset value can also be other values, such as 1 / 2, 1 / 3, or 3 / 4, etc., or there may be no need for this preset value restriction.

[0016] Referring to Figure 2, the database 13 stores multiple sets of data 13-1, 13-2...13-X. Each set of data may include one or more insect classification information 131 and corresponding acoustic features 132. Both the insect classification information 131 and the corresponding acoustic features 132 are known, meaning that the data is pre-established. The insect classification information 131 includes information such as the insect species, scientific name, subspecies, survival status, and appearance. The acoustic features 132 are related physical quantities of the insect's wingbeat sound, including but not limited to sound pressure, sound pressure level, amplitude, sound energy, frequency intensity, digital audio sample value, or voltage. In one example, the selection of the acoustic property is determined based on the acoustic feature 132 of the data in the database 13. For example, if the acoustic feature 132 of the data in the database 13 is frequency, then the sound data is converted into converted data containing frequency.

[0017] If the salient feature of the acoustic property in the converted data matches at least one of the acoustic features 132 in the database 13, then the processing unit 12 generates a comparison result based on the match. The processing unit 12 may have an artificial intelligence module that can perform an acoustic matching algorithm to find the associated acoustic features 132 based on the sound data and make a judgment to obtain the comparison result.

[0018] Figure 3 shows a system block diagram of another example of the present invention. The insect identification system includes an electronic device 20, which comprises a sound receiving unit 21, a conversion unit 22, a filtering unit 23, and a processing unit 24. The electronic device 20 is communicatively connected to a database 30. The conversion unit 22 is coupled to the sound receiving unit 21. The conversion unit 22 can convert the sound signal from analog to digital, and can also convert the sound signal to other formats or forms of signals or data. The conversion unit 22 outputs an output signal to the filtering unit 23, which performs one or more processing operations on the output signal. In one example, the processing removes the portion of the sound signal other than the wing vibration sound. The processing unit 24 is coupled to the filtering unit 23. It should be understood that the conversion unit 22 and the filtering unit 23 may be integrated with the sound receiving unit 21 into a module, or they may be integrated into the processing unit 24.

[0019] The following example uses the fall armyworm as an illustration. Before identification, the acoustic characteristics of the fall armyworm must be established. First, the wingbeat frequencies of the fall armyworm under different conditions were detected. The number of samples in this experiment was 6 groups. Figure 4 shows the frequency distribution under different amplitudes. The darker the color, the more frequencies were detected. The wingbeat frequencies are mainly distributed between 2.34 Hz ​​and 3.125 Hz and between 35 Hz and 49 Hz, which can be summarized in Table 1 below (the former is not shown in Table 1 below).

[0020] Table 1 [variety] [gender] [Temperature (°C)] [Age in days] [(] [sky] [)] [Wing vibration frequency] Fall armyworm male twenty three 1 35 Hz -37 Hz Fall armyworm male twenty four 1 44 Hz -49 Hz Fall armyworm female twenty four 1 32 Hz -35 Hz Fall armyworm female 24.5 2 48 Hz - 50 Hz Fall armyworm female 24.5 2 39 Hz -49 Hz Fall armyworm male 24.5 2 35 Hz -42 Hz

[0021] The aforementioned data can be imported into database 13 as one set of data. Similarly, samples can be taken from known insects to obtain database 13 containing multiple sets of data.

[0022] In application, the sound receiving unit 11 is placed in a detection area, such as a field area planted with crops. The sound receiving unit 11 receives a sound signal from the surrounding area of ​​the field area. Figure 5A shows a schematic diagram of the time-domain waveform of the sound signal in this example. The horizontal axis is time (Sec), and the vertical axis is sound pressure, with the unit being pressure (Pa). The sound receiving unit 11 of the insect identification system 10 samples the sound signal within a sampling time t. This example is conducted in an experimental manner, so the sound signal is all or almost all of the wing flapping of the target insect. Based on the wing flapping behavior of the target insect, the sound signal includes one or more first time intervals t1, t2 with sound (i.e., the part with obvious amplitude) and one or more second time intervals t3, t4, t5 without sound (i.e., the part without obvious amplitude). This is an example for illustration. In some examples, depending on the sampling conditions, the sampling time t may be entirely the first time interval with sound or entirely the second time interval without sound. Returning to Figure 5A, for the first time intervals t1 and t2 where there is sound, one or more features are reflected in the waveform. These features are those where the frequency waveform meets certain preset conditions within the sampling time t, such as the amplitude of the sound signal waveform or the portion of its variation that is greater than a threshold value.

[0023] Figure 5B is a spectrum diagram of Figure 5A. The horizontal axis represents sound frequency (Hz), and the vertical axis represents the root mean square value of sound pressure (Pa RMS), showing the sound pressure energy distribution at different frequencies. From the maximum peak P1 and the second largest peak P2 in Figure 5B, it can be seen that the sound frequencies (Hz) in Figure 5A are mostly 2.34 Hz ​​and 4.69 Hz. As mentioned above, in Figure 5A, the sum of the first time intervals t1 and t2 (i.e., the audible interval) is less than the preset value (i.e., 2 / 3). Therefore, further processing of the time-domain waveform diagram data is required. This example extracts a portion 40 of the time-domain waveform diagram, which is shown in Figure 6A. The sampling time t' is a segment within the sampling time t. The proportion of the first time interval t2 (i.e., the audible interval) in the sampling time t' increases, exceeding the preset value (i.e., 2 / 3), and therefore can be used as a basis for analysis and identification. Figure 6B is the spectrum diagram of Figure 6A. From the maximum peak P1' and the second largest peak P2' in Figure 6B, it can be seen that the sound frequencies (Hz) of Figure 6A are mostly 2.34 Hz ​​and 37.5 Hz.

[0024] By comparing and identifying the results of Figures 6A and 6B with the database 13 established according to Table 1, a matching result can be obtained. Therefore, it can be determined that the target insect is Fall Armyworm.

[0025] Referring to Figure 7, the present invention can also transmit the matching result to an external device 50 (such as a mobile phone or personal computer), and display the result associated with the insect classification information 131 (such as the insect species, scientific name, subspecies, survival status, and appearance) through a display unit 51 of the external device 50 for the user to refer to.

[0026] In summary, this invention converts the detected sound signal into a signal containing acoustic properties, and then compares it with wing vibration sound information in the database to obtain results associated with the insect classification information. Compared with identification methods based on images or appearance, the system and method of this invention do not require insect capture, making them more convenient. They are also not limited to a static state and can be used for identification whether the insect is stationary or in flight, requiring less environmental control. Furthermore, by defining salient features of the converted signal and matching them with the wing vibration sound information, the accuracy of the matching can be further increased. The system and method of this invention can be widely applied in experimental sites or fields to improve the identification and prediction results of insect species. They can also be used to analyze insect species density, aiding in the control of field pests, improving pest prevention effectiveness, reducing pesticide use, and solving the problem that conventional techniques cannot identify insects without chirping sounds using acoustic methods.

[0027] 10: Insect Identification System 11: Sound receiving unit 12: Processing Unit 13: Database 13-1, 13-2, 13-X: Data 131: Insect Classification Information 132: Acoustic characteristics 20: Electronic devices 21: Sound receiving unit 22: Conversion Unit 23: Filtering Unit 24: Processing Unit 30: Database 40: Part 50: External devices 51: Display Unit t, t': Sampling time t1, t2: First time interval t3, t4, t5: Second time interval P1, P1': Maximum peak value P2, P2': Second largest peak value

Claims

1. A sound-based insect identification system, comprising: A sound receiving unit receives a sound signal; A processing unit is coupled to the sound receiving unit, and the processing unit obtains sound data based on the sound signal; The system includes a database storing multiple sets of data, including multiple insect classification information reflecting known insect species and multiple wing vibration sound information corresponding to the insect classification information. The sound data is converted into transformed data containing acoustic properties, and the acoustic properties in the transformed data are defined by one or more salient features. These salient features are compared with the multiple wing vibration sound information in the data. If the salient feature matches at least one wing vibration sound information, the sound data is associated with the insect classification information corresponding to the matching wing vibration sound information. The sound receiving unit receives the sound signal at a sampling time. The sound data reflects the change of the sound signal over time. The sampling time consists of one or more audible intervals and one or more silent intervals. The ratio of the sum of the audible intervals to the sampling time is not less than a preset value before the comparison is performed.

2. The insect identification system as claimed in claim 1, wherein the sound signal includes the sound of one of the insect's wings vibrating, and before the comparison is performed, sounds other than the wing vibration sound are removed from the sound signal.

3. The insect identification system as claimed in claim 2, wherein the sound signal includes the sound of one wing vibration and a chirping sound of the insect, and the comparison does not involve the chirping sound.

4. The insect identification system as described in claim 4, wherein the preset value is 2 / 3.

5. The insect identification system as claimed in claim 1, wherein the sound data is a time-domain waveform, and the converted data is a spectrum of the time-domain waveform, wherein the salient feature is the main frequency component of the spectrum.

6. A sound-based insect identification method, comprising the following steps: receiving a sound signal in a detection area, the reception of the sound signal lasting for a sampling time; obtaining sound data based on the sound signal, the sound data reflecting the change of the sound signal over time during the sampling time; converting the sound data into transformed data containing acoustic properties, the acoustic properties in the transformed data being defined as one or more salient features; and comparing the salient features with a complex set of data to obtain a comparison result, the data including multiple insect classification information reflecting known insect species and multiple wing vibration sound information corresponding to the insect classification information; wherein, If the significant feature matches at least one of the wing vibration sound information, the comparison result is that the sound data is associated with the insect classification information corresponding to the matched wing vibration sound information. The sampling time consists of one or more sound intervals and one or more silent intervals. The ratio of the sum of the sound intervals to the sampling time is not less than a preset value before the comparison is performed.

7. The insect identification method as claimed in claim 7, wherein the sound signal includes the sound of one of the insect's wings vibrating and a chirping sound, and the comparison does not involve the chirping sound.

8. The insect identification method as described in claim 7, wherein the sound data is a time-domain waveform, and the converted data is a spectrum of the time-domain waveform, wherein the salient feature is the main frequency component of the spectrum.

Images