Food intake intensity detection method, related device, equipment and storage medium

Abstract

The invention discloses an ingestion intensity detection method, a related device, equipment and a storage medium, and the method comprises the steps: detecting the ingestion audio of a target water product in a target water area based on an ingestion detection model, and obtaining first ingestion intensity; wherein the ingestion detection model is obtained based on training of a sample audio set; obtaining a second feeding intensity after the first feeding intensity is checked; on the basis of ingestion audio which is marked with second ingestion intensity and serves as sample audio, updating the sample audio set, and on the basis of a checking statistical result of the ingestion intensity, obtaining a model analysis result representing whether the current growth stage of the target water product is adaptive to the ingestion detection model or not; and based on a model analysis result, determining whether to perform optimization training on the ingestion detection model based on the sample audio set. According to the scheme, the accuracy and the stability of detecting the ingestion intensity can be improved, especially in a deep and far sea culture scene.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for detecting feeding intensity and related devices, equipment and storage media. Background Art

[0002] In aquaculture, especially deep-sea aquaculture, accurate feeding of bait is an important link that cannot be ignored. Among them, assessing the feeding intensity is an extremely important key point in accurate feeding of bait.

[0003] At present, computer vision-based methods are usually used to evaluate feeding intensity. However, this method is usually based on natural conditions such as clear water, sufficient light, and stable background. Otherwise, the detection accuracy of feeding intensity will be greatly affected. In view of this, how to improve the accuracy and stability of feeding intensity detection, especially in deep-sea aquaculture scenarios, has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a feeding intensity detection method and related devices, equipment and storage media, which can improve the accuracy and stability of feeding intensity detection, especially in deep-sea aquaculture scenarios.

[0005] In order to solve the above-mentioned technical problems, the first aspect of the present application provides a feeding intensity detection method, including: detecting the feeding audio of the target aquatic product in the target water area based on a feeding detection model to obtain a first feeding intensity; wherein the feeding detection model is trained based on a sample audio set; obtaining a second feeding intensity after the first feeding intensity is calibrated; based on the feeding audio marked with the second feeding intensity and used as the sample audio, updating the sample audio set, and based on the calibration statistical results of the feeding intensity, obtaining a model analysis result characterizing whether the current growth stage of the target aquatic product is suitable for the feeding detection model; based on the model analysis result, determining whether to optimize the feeding detection model based on the sample audio set.

[0006] In order to solve the above-mentioned technical problems, the second aspect of the present application provides a feeding intensity detection device, including: a feeding detection module, an intensity verification module, an update analysis module and an optimization training module, the feeding detection module is used to detect the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain a first feeding intensity; wherein the feeding detection model is obtained by training based on the sample audio set; the intensity verification module is used to obtain a second feeding intensity after the first feeding intensity is verified; the update analysis module is used to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and obtain a model analysis result based on the verification statistics of the feeding intensity to characterize whether the current growth stage of the target aquatic product is suitable for the feeding detection model; the optimization training module is used to determine whether to optimize the feeding detection model based on the sample audio set based on the model analysis result.

[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, which at least includes a memory and a processor coupled to each other, the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the food intake intensity detection method in the first aspect.

[0008] In order to solve the above technical problem, the fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, and the program instructions are used to implement the above-mentioned feeding intensity detection method of the first aspect.

[0009] In the above scheme, the feeding audio of the target aquatic product in the target water area is detected based on the feeding detection model to obtain a first feeding intensity, and the feeding detection model is trained based on the sample audio set, and then a second feeding intensity after the first feeding intensity is verified is obtained, so as to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and based on the verification statistics of the feeding intensity, a model analysis result is obtained to characterize whether the current growth stage of the target aquatic product is suitable for the feeding detection model, and based on the model analysis result, it is determined whether to perform the feeding detection based on the sample audio set. The test model is optimized and trained. Therefore, on the one hand, it is different from using computer vision technology and using audio data for feeding detection. It can avoid the influence of natural conditions such as whether the water is clear, whether the light is sufficient, and whether the background is stable as much as possible on feeding detection, and only needs to pay attention to the feeding sound, which is usually a stable existence of aquatic products during the feeding process. On the other hand, since the feeding verification is also statistically analyzed during the feeding detection process, and it is determined whether to update the training feeding detection model based on this, the applicability of the feeding detection model can be synchronized with the growth stage of aquatic products as much as possible. Therefore, the accuracy and stability of detecting feeding intensity can be improved, especially in deep-sea aquaculture scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of an embodiment of a method for detecting food intake intensity of the present application; Figure 2 is a schematic diagram of a framework of an embodiment of a food ingestion detection model; Figure 3 is a process schematic diagram of an embodiment of the method for detecting the intensity of food intake of the present application; Figure 4 It is a schematic diagram of the framework of an embodiment of a food intake intensity detection device of the present application; Figure 5 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application; Figure 6 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0011] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0012] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0013] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0014] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the method for detecting the intensity of food intake of the present application. Specifically, the method may include the following steps: Step S11: Detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain a first feeding intensity.

[0015] In the embodiment of the present disclosure, the feeding detection model is trained based on the sample audio set. It should be noted that the sample audio set may include feeding audio collected during the feeding process of the target aquatic product. For the convenience of distinction, after the feeding audio is included in the sample audio set, it can also be called sample audio. The sample audio in the sample audio set can also be marked with feeding intensity. For the convenience of distinction, the feeding intensity marked with the sample audio can be called sample feeding intensity. In addition, as a possible example, the specific intensity of "feeding intensity" such as "first feeding intensity", "second feeding intensity", "sample feeding intensity" in the embodiment of the present disclosure can be any one of several preset feeding intensities. Exemplarily, several preset feeding intensities may include but are not limited to: no feeding, weak feeding, medium feeding, strong feeding, etc. Of course, the above example is only a possible example of several feeding intensities in actual application, and the specific content of several feeding intensities is not limited thereto. For example, several feeding intensities can also be divided more finely or more coarsely, which is not limited here.

[0016] In one implementation scenario, the target aquatic products may include, but are not limited to, fish, shellfish, shrimp, crab, turtle, frog, etc. The specific types of the target aquatic products are not limited herein.

[0017] In one implementation scenario, the acoustic features of the feeding audio (such as Fbank, MFCC, etc.) can be extracted, and then the acoustic features of the feeding audio can be input into the feeding detection model, so that the first feeding intensity output by the feeding detection model can be obtained. It should be noted that the feeding detection model may include but is not limited to a convolutional neural network, etc., and the network architecture of the feeding detection model is not limited here. In order to improve the accuracy of the feeding detection model, the feeding detection model can also be trained in advance. As a possible example, a sample audio set can be obtained in advance, and the sample audio set as the sample audio is the feeding audio collected for the target aquatic product, and the sample audio can be marked with the sample feeding intensity. On this basis, the sample audio in the sample audio set can be detected based on the feeding detection model to obtain the predicted feeding intensity of the sample audio (specifically, the acoustic features of the sample audio can be input into the feeding detection model to obtain the predicted feeding intensity output by the feeding detection model), so that the difference between the sample feeding intensity and the predicted feeding intensity can be measured by a loss function such as cross entropy, and the training loss of the feeding detection model can be obtained, and the network parameters of the feeding detection model can be adjusted based on the training loss. It should be noted that for the specific measurement of the loss function, please refer to the technical details of the loss function such as cross entropy, which will not be repeated here.

[0018] In a specific implementation scenario, as a possible example, the target aquatic product feeding sound signal data can be collected through N hydrophones carried by the aquaculture cage platform within the time range of the target aquatic product feeding, and stored as feeding audio (in the training stage, it can be used as sample feeding audio) after amplification, filtering, and format conversion by the signal conditioner. In addition, in order to facilitate data labeling of sample feeding audio, when conditions permit, the feeding video signal can also be synchronously recorded as auxiliary information to form a sample audio set: …… (1) In the above formula (1), represents a sample audio set, represents the i-th sample feeding audio in the sample audio set, Represents the total number of sample feeding audios in the sample audio set. Of course, the above example is only a possible example of obtaining feeding audio, and it does not limit the collection of feeding audio by other methods, and no further examples are given here. It should be noted that the initially collected sample audio set can be regarded as the basic data for the initial training of the feeding detection model, and the subsequently updated sample audio set can be regarded as the updated data for optimizing and updating the feeding detection model. For details, please refer to the subsequent related descriptions, which will not be repeated here.

[0019] In a specific implementation scenario, in order to facilitate data annotation of sample feeding audio, a knowledge base on feeding intensity of target aquatic products can also be pre-constructed, and the knowledge base can define definition information of target aquatic products on various feeding intensities. Taking the target aquatic product as fish and the feeding intensity as the four levels of intensity as mentioned above as an example, the definition information of "strong feeding" can include but is not limited to: "The fish school reacts strongly to food, gathers near the feeding point, and the feeding behavior produces relatively loud sounds", "Medium feeding" definition information can include but is not limited to: "The fish school reacts strongly to food, has gathering behavior, and the feeding behavior produces certain sounds", "Weak feeding" definition information can include but is not limited to: "Weak reaction to food, only a small part of the feeding behavior occurs, the sound of the fish school is weak, and there are slight fluctuations", "No feeding" definition information can include but is not limited to: "The fish school has no reaction to food, and the sound of the fish school is weak and stable". Of course, the above knowledge base is only a possible example in the actual application process, and other cases can be deduced by analogy, and no examples are given here.

[0020] In another implementation scenario, different from the aforementioned implementation of the feeding detection model, as another possible implementation of the feeding detection model, characteristic spectrograms of several characteristic types of feeding audio can also be extracted, and the sub-weights of various characteristic spectrograms in the same frequency band can be obtained, and then based on the sub-weights of various characteristic spectrograms in the same frequency band, the sub-spectragrams of various characteristic spectrograms in the corresponding frequency band are weighted to obtain a weighted spectrogram, and prediction is performed based on the weighted spectrogram to obtain the first feeding intensity. The above method refers to the characteristic spectrograms of several characteristic types and weights various characteristic spectrograms in the same frequency band to predict the feeding intensity accordingly. Therefore, it can not only integrate the advantages of various characteristic spectrograms in capturing feeding activity information, but also perform spectrum fusion in the frequency band dimension, which helps to improve the granularity of spectrum fusion, and further improve the detection accuracy of feeding intensity.

[0021] In a specific implementation scenario, taking the characteristic spectra of several characteristic types including the CQT (Constant-Q Transform) spectrum as an example, the CQT spectrum can be expressed as: …… (2) In the above formula (2), Is the length , Q is the constant factor in CQT, k is the frequency number of the CQT spectrum. For details, please refer to the technical details of CQT, which will not be repeated here.

[0022] In a specific implementation scenario, taking the characteristic spectra of several characteristic types including the STFT (Short Time Fourier Transform) spectrum as an example, the STFT spectrum can be expressed as: …… (3) In the above formula (3), It is a window function. By adjusting the position of the window, the signal is intercepted on the time axis for Fourier transform, so as to achieve the purpose of short-time Fourier transform spectrum analysis. For details, please refer to the technical details of STFT, which will not be repeated here.

[0023] In a specific implementation scenario, taking the characteristic spectra of several feature types including MFCC (Mel Frequency Cepstral Coefficient) spectra as an example, the specific extraction process includes: preprocessing, Fourier transform, Mel filter bank, logarithmic operation, discrete cosine transform and dynamic feature extraction. Among them, the most critical is the design of the Mel filter bank, and each filter function can be expressed as: …… (4) In the above formula (4), is the center frequency of each filter. For details, please refer to the technical details of MFCC, which will not be repeated here.

[0024] In a specific implementation scenario, taking the characteristic spectrograms of several characteristic types including CQT spectrogram, STFT spectrogram and MFCC spectrogram as examples, the characteristic spectrograms of several characteristic types of food ingestion audio can be uniformly expressed as: …… (5) In the above formula (5), Characteristic spectrogram representing several characteristic types of food ingestion audio, CQT spectrogram representing the ingested audio, The STFT spectrogram representing the ingested audio, MFCC spectrogram representing the ingestion audio. In addition, the sample ingestion audio with unclear feature expression and quality problems in the sample audio set can also be cleaned.

[0025] In a specific implementation scenario, in order to measure the sub-weights of various characteristic spectra in the same frequency band, the energy intensity of various characteristic spectra in the same frequency band can be obtained, and then each characteristic spectrum is selected as the current spectrum, and a characteristic spectrum other than the current spectrum is selected as a reference spectrum, so that the sub-weight of the current spectrum in the corresponding frequency band can be obtained based on the ratio of the energy intensity of the current spectrum and the reference spectrum in the same frequency band. It should be noted that when there are multiple characteristic spectra other than the current spectrum, the sub-weight of the current spectrum in the corresponding frequency band can be obtained based on the ratio of the energy intensity of the current spectrum in a specific frequency band and the sum of the energy intensities of each reference spectra in the corresponding frequency band. For the sake of ease of description, the characteristic spectra of several characteristic types, including CQT spectra, STFT spectra and MFCC spectra, are still taken as examples. In the i-th frequency band The sub-weights of the CQT spectrogram, STFT spectrogram and MFCC spectrogram can be expressed as: …… (6) …… (7) …… (8) In the above formulas (6) to (8), , , Respectively represent the CQT spectrum, STFT spectrum and MFCC spectrum in the frequency band The energy intensity on , , Respectively represent the CQT spectrum, STFT spectrum and MFCC spectrum in the frequency band On this basis, the CQT spectrogram, STFT spectrogram and MFCC spectrogram have sub-weights in each frequency band, which can be expressed as a weight matrix: …… (9) In the above formula (9), represents the weight matrix, , , Respectively represent the weight matrices of the CQT spectrogram, STFT spectrogram and MFCC spectrogram in the weight matrix. It should be noted that the above example is only a possible example of obtaining the sub-weights of various characteristic spectrograms in the same frequency band, and it is not limited to other ways to obtain the sub-weights. For example, the sub-weight of the current spectrum in the corresponding frequency band can also be obtained based on the ratio of the energy intensity of the current spectrum and various characteristic spectrograms in the same frequency band, and no examples are given here one by one.

[0026] In one implementation scenario, as a possible example, see Figure 2 , Figure 2 Schematic diagram of a framework of an embodiment of a food ingestion detection model. Figure 2As shown, the feeding detection model can specifically adopt the design architecture of "feature extractor + feature fusion device + classifier" cascade coupling. Among them, the implementation process of "feature extractor + feature fusion device" can refer to the above-mentioned related description, which will not be repeated here. Of course, in the implementation process, it is not limited to this. For example, after extracting the feature spectra of several feature types, the feature spectra of several feature types can be first input into a network model such as a convolutional neural network for feature extraction, and then in the feature extraction process, refer to the above-mentioned description of feature fusion for feature fusion, and the fused weighted spectra will continue to be processed by the subsequent network layer until it reaches the classification layer. The classification layer can divide the feature mapping range according to the mapping relationship between the learned features and the feeding intensity, thereby classifying a specific feeding audio as intensity. As a possible example, the evaluation and grading of feeding intensity can be realized based on a network such as ResNet-34. The feature data after feature fusion is used as the input of ResNet-34. After the data dimension is adjusted, the 7*7 convolution layer is first connected to the ResNet-34 network, and the convolution layer is connected to the batch normalization layer, and then the 3*3 maximum pooling downsampling layer. Then, 4 modules composed of residual blocks can be used. Among them, Conv2_x uses 3 layers of residual layers, Conv3_x uses 4 layers of residual layers, Conv4_x uses 6 layers of residual layers, and Conv5_x uses 3 layers of residual layers. Finally, the number of output channels is changed to 4 through the fully connected layer, and the probability values ​​of various feeding intensities can be output, such as the probability values ​​of "strong feeding", "medium feeding", "weak feeding" and "no feeding", so as to realize the evaluation and grading of feeding intensity. It should be noted that the above example is only a possible architecture of the feeding detection model, and the specific structure of the feeding detection model is not limited here.

[0027] Step S12: obtaining a second food intake intensity after the first food intake intensity has been calibrated.

[0028] In one implementation scenario, the feeding detection model can be deployed on the application side. After obtaining the first feeding intensity of the feeding audio, the application side can upload the feeding audio and its first feeding intensity to the upper management system, so that the upper management system can verify the first feeding intensity of the feeding audio. The second feeding intensity of the feeding audio is obtained. In addition, when the feeding audio and the feeding video are collected at the same time, the feeding video can also be uploaded to the upper management system together with the feeding audio and its first feeding intensity, so that the upper management system can verify the first feeding intensity of the feeding audio with reference to the feeding video. In addition, after the upper management system verifies the first feeding intensity, the second feeding intensity of the feeding audio after verification can be fed back to the application side. As a possible example, the upper management system can be manned by relevant personnel with professional knowledge of aquatic products, so that the relevant personnel can verify the first feeding intensity.

[0029] In an implementation scenario, the second feeding intensity after verification may be the same as the first feeding intensity, or the second feeding intensity after verification may be different from the first feeding intensity, which is not limited here.

[0030] Step S13: Based on the feeding audio marked with the second feeding intensity and used as the sample audio, the sample audio set is updated, and based on the verification statistical results of the feeding intensity, a model analysis result is obtained to characterize whether the current growth stage of the target aquatic product is suitable for the feeding detection model.

[0031] In an implementation scenario, after the second eating intensity of the eating audio is obtained, the second eating intensity can be marked on the eating audio, and the eating audio marked with the second eating intensity is used as the sample audio to update the sample audio set based on it.

[0032] In a specific implementation scenario, in order to update the sample audio set, the sample audio before the most recent optimization training of the food ingestion detection model in the sample audio set may be deleted. Exemplarily, as mentioned above, the sample audio set for the initial training of the food ingestion detection model (for the sake of distinction, it may be referred to as basic data) may include sample audio to After the initial training, the feeding detection model can be used to continue to detect the feeding intensity and the aforementioned related steps for the newly collected feeding audio until the current growth stage of the target aquatic product is found to be unsuitable for the feeding detection model for the first time. At this time, the newly accumulated sample audio to , then the sample audio of the feeding detection model before the first optimization training in the sample audio set can be deleted, that is, the sample audio to That is, the updated sample audio set (for the sake of distinction, it can be called update data) can contain sample audio to Of course, the above example is only a possible example, and other cases can be deduced by analogy, which will not be listed here one by one. For example, you can delete only the sample audio to Specifically, the sample audios in the sample audio set before the most recent optimization training of the self-feeding detection model can be sorted in order of collection time from earliest to latest, and the sample audios before the preset order (such as the first 50%, the first 70%, etc.; or, the first 100%, that is, all) can be deleted. The above method deletes at least part of the sample audios in the sample audio set before the most recent optimization training of the self-feeding detection model, which can eliminate the sample audios in the sample audio set that are not suitable for the current growth stage as much as possible, which helps to improve the adaptability of the feeding detection model after optimization training to the current growth stage.

[0033] In a specific implementation scenario, different from the above implementation, in order to update the sample audio set, the sample audio in the sample audio set before the most recent optimization training of the food ingestion detection model can also be retained. Exemplarily, as mentioned above, the sample audio set for the initial training of the food ingestion detection model (for the sake of distinction, it can be referred to as basic data) can include sample audio to After the initial training, the feeding detection model can be used to continue to detect the feeding intensity and the aforementioned related steps for the newly collected feeding audio until the current growth stage of the target aquatic product is found to be unsuitable for the feeding detection model for the first time. At this time, the newly accumulated sample audio to , the sample audio before the most recent optimization training of the self-feeding detection model in the sample audio set can be retained. In other words, the sample audio set after the update contains sample audio to And sample audio to As a possible example, in this case, when optimizing the feeding detection model using the updated sample audio set, the sample audio can be set to The learning rate is lower than the sample audio to The above method retains the sample audio before the most recent optimization training of the feeding detection model in the sample audio set, and can refer to the feeding audio up to the current growth stage while changing with the growth stage of the target aquatic product, which helps the feeding detection model to more comprehensively support feeding detection at the growth stage level.

[0034] It should be noted that the above examples are only two possible examples of updating the sample audio set. In actual application, the sample audio set can be updated according to specific circumstances, and is not limited to the above two examples. In addition, the specific method of updating the sample audio set can also be adaptively selected according to actual circumstances. For example, in the case where the feeding performance of the target aquatic product varies greatly with the growth stage, the former of the above two methods can be selected, or, in the case where the feeding performance of the target aquatic product varies relatively little with the growth stage, the latter of the above two methods can be selected.

[0035] In an implementation scenario, in order to obtain the model analysis results, each feeding audio after the most recent optimization training of the self-feeding detection model can be selected as a historical audio, and based on the calibration results of each historical audio about the feeding intensity, the calibration statistical results are obtained, and the calibration results include: the historical audio is a first audio whose first feeding intensity is consistent with the second feeding intensity, or the historical audio is a second audio whose first feeding intensity is inconsistent with the second feeding intensity, and the calibration statistical results include at least one of the following: a first proportion of the first audio, and a second proportion of the second audio. On this basis, the model analysis results can be obtained based on the calibration statistical results. The above method, by statistically analyzing the calibration results of each historical audio about the feeding intensity after the most recent optimization training of the self-feeding detection model, can evaluate and analyze the model according to historical data after each calibration, which helps to timely discover whether the model is adapted to the growth stage.

[0036] In a specific implementation scenario, as described above, the sample audio set (for the sake of distinction, it can be referred to as basic data) for initially training the food ingestion detection model can include sample audio to After the initial training, the feeding detection model can be used to continue to detect the feeding intensity and the aforementioned related steps for the newly collected feeding audio. In this process, the feeding audio can be accumulated. , , Etc. For example, when the currently collected food intake audio is In the case of feeding audio to Of course, the above example is only a possible example in the actual application process, and other cases can be deduced by analogy, so I will not give examples one by one here.

[0037] In a specific implementation scenario, still taking the above example as an example, the food ingestion audio can be obtained respectively. to The calibration result is to determine whether the first feeding intensity predicted by the feeding detection model is consistent with the second feeding intensity after calibration. On this basis, the number of feeding audios (i.e., the first audio) that are consistent with the first feeding intensity and the second feeding intensity can be counted in the feeding audio. to and / or, counting the first proportion of the first feeding intensity and the second feeding intensity that are inconsistent with the feeding audio (ie, the second audio) in the feeding audio to Of course, the above example is only a possible example in the actual application process, and other cases can be deduced by analogy, so I will not give examples one by one here.

[0038] In a specific implementation scenario, when the verification statistical result includes the first proportion, the model analysis result can be obtained based on the size comparison result between the first proportion and the first threshold. Exemplarily, in response to the first proportion being not less than the first threshold, it can be determined that the model analysis result includes that the current growth stage of the target aquatic product is adapted to the feeding detection model; conversely, in response to the first proportion being less than the first threshold, it can be determined that the model analysis result includes that the current growth stage of the target aquatic product is not adapted to the feeding detection model. It should be noted that the specific value of the first threshold can be set according to the detection accuracy of the feeding intensity. Exemplarily, when the detection accuracy requirement for the feeding intensity is relatively high, the first threshold can be set larger, or, when the detection accuracy requirement for the feeding intensity is relatively loose, the first threshold can be set slightly smaller. The specific value of the first threshold is not limited here. In the above manner, when the verification statistical result includes the first proportion, the model analysis result is determined by obtaining the size comparison result between the first proportion and the first threshold, and the model analysis result can be determined by numerical comparison, which helps to simplify the evaluation complexity of the feeding detection model.

[0039] In a specific implementation scenario, when the verification statistical result includes the second proportion, the model analysis result can be obtained based on the size comparison result between the second proportion and the second threshold. Exemplarily, in response to the second proportion being not less than the second threshold, it can be determined that the model analysis result includes that the current growth stage of the target aquatic product is not suitable for the feeding detection model; conversely, in response to the second proportion being lower than the second threshold, it can be determined that the model analysis result includes that the current growth stage of the target aquatic product is suitable for the feeding detection model. It should be noted that the specific value of the second threshold can be set according to the detection accuracy of the feeding intensity. Exemplarily, in the case where the detection accuracy requirement for the feeding intensity is relatively high, the second threshold can be set smaller, or, in the case where the detection accuracy requirement for the feeding intensity is relatively loose, the second threshold can be set slightly larger. The specific value of the second threshold is not limited here. In the above manner, when the verification statistical result includes the second proportion, the model analysis result is determined by obtaining the size comparison result between the second proportion and the second threshold, and the model analysis result can be determined by numerical comparison, which helps to simplify the evaluation complexity of the feeding detection model.

[0040] Step S14: Based on the model analysis result, determine whether to optimize the food ingestion detection model based on the sample audio set.

[0041] In one implementation scenario, in response to the model analysis result indicating that the current growth stage is not suitable for the feeding detection model, the feeding detection model can be optimized and trained based on the sample audio set. For the specific training process of the feeding detection model, please refer to the above-mentioned related description, which will not be repeated here. Figure 3 , Figure 3 Schematic diagram of the process of an embodiment of the method for detecting the intensity of food intake of the present application. Figure 3 As shown, after optimizing and training the feeding detection model, the step of detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model can be returned to obtain the first feeding intensity. By repeating in this way, the feeding detection model can be optimized and trained synchronously following the full growth cycle of the target aquatic product, which helps to realize adaptive feeding detection in the full growth cycle of the target aquatic product, so as to maintain the stability of feeding intensity detection in the full growth cycle.

[0042] In one implementation scenario, in response to the model analysis results characterizing that the current growth stage is suitable for the feeding detection model, the step of detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain the first feeding intensity can be returned, and the current feeding detection model can be continued to be used for feeding detection in the current growth stage, which helps to maintain the stability of feeding intensity detection in the feeding detection results of the current growth stage.

[0043] It should be noted that the above-mentioned cycle iteration can continue until the target aquatic product in the target waters is caught, at which time there is no need to detect the feeding intensity; or, when the target aquatic product in the target waters is caught and the target aquatic product of the same species is re-introduced, the above-mentioned description can be referred to, and the feeding detection model can be initially trained using the basic data and deployed on the application side. Subsequently, the above-mentioned description can be referred to based on the statistical results of the verification of the feeding audio about the feeding intensity to determine whether to optimize the feeding detection model; or, when the target aquatic product in the current target waters is caught and a new target aquatic product (different from the original species) is re-introduced, the above-mentioned description can be referred to, and the basic data of the new target aquatic product can be initially trained on the feeding detection model, and deployed on the application side. Subsequently, the above-mentioned description can be referred to based on the statistical results of the verification of the feeding audio about the feeding intensity to determine whether to optimize the feeding detection model. The above examples are only a few possible examples in the actual application process, and other possible situations are not limited thereto, and no examples are given one by one here.

[0044] In the above scheme, the feeding audio of the target aquatic product in the target water area is detected based on the feeding detection model to obtain a first feeding intensity, and the feeding detection model is trained based on the sample audio set, and then a second feeding intensity after the first feeding intensity is verified is obtained, so as to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and based on the verification statistics of the feeding intensity, a model analysis result is obtained to characterize whether the current growth stage of the target aquatic product is suitable for the feeding detection model, and based on the model analysis result, it is determined whether to perform the feeding detection based on the sample audio set. The test model is optimized and trained. Therefore, on the one hand, it is different from using computer vision technology and using audio data for feeding detection. It can avoid the influence of natural conditions such as whether the water is clear, whether the light is sufficient, and whether the background is stable as much as possible on feeding detection, and only needs to pay attention to the feeding sound, which is usually a stable existence of aquatic products during the feeding process. On the other hand, since the feeding verification is also statistically analyzed during the feeding detection process, and it is determined whether to update the training feeding detection model based on this, the applicability of the feeding detection model can be synchronized with the growth stage of aquatic products as much as possible. Therefore, the accuracy and stability of detecting feeding intensity can be improved, especially in deep-sea aquaculture scenarios.

[0045] See also Figure 4 , Figure 4 It is a schematic diagram of the framework of an embodiment of the feeding intensity detection device of the present application. The feeding intensity detection device 40 includes: a feeding detection module 41, an intensity verification module 42, an update analysis module 43 and an optimization training module 44. The feeding detection module 41 is used to detect the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain the first feeding intensity; wherein the feeding detection model is obtained by training based on the sample audio set; the intensity verification module 42 is used to obtain the second feeding intensity after the first feeding intensity is verified; the update analysis module 43 is used to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and obtain the model analysis result characterizing whether the current growth stage of the target aquatic product is suitable for the feeding detection model based on the verification statistics of the feeding intensity; the optimization training module 44 is used to determine whether to optimize the feeding detection model based on the sample audio set based on the model analysis result.

[0046] In the above scheme, the feeding intensity detection device 40 detects the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain a first feeding intensity, and the feeding detection model is obtained based on the training of the sample audio set, and then obtains the second feeding intensity after the first feeding intensity is verified, so as to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and obtains the model analysis result characterizing whether the current growth stage of the target aquatic product is suitable for the feeding detection model based on the statistical result of the feeding intensity, and determines whether the feeding detection model is suitable for the current growth stage of the target aquatic product based on the sample audio based on the model analysis result. The feeding detection model is optimized and trained by the set. Therefore, on the one hand, it is different from the use of computer vision technology and the use of audio data for feeding detection. It can avoid the influence of natural conditions such as whether the water is clear, whether the light is sufficient, and whether the background is stable as much as possible on the feeding detection, and only needs to pay attention to the feeding sound, which is usually a stable existence of aquatic products during the feeding process. On the other hand, since the feeding verification is also statistically analyzed during the feeding detection process, and it is determined whether to update the training feeding detection model based on this, the applicability of the feeding detection model can be synchronized with the growth stage of aquatic products as much as possible. Therefore, the accuracy and stability of detecting feeding intensity can be improved, especially in deep-sea aquaculture scenarios.

[0047] In some disclosed embodiments, the optimization training module 44 includes an optimization training submodule for optimizing the feeding detection model based on the sample audio set in response to the model analysis result indicating that the current growth stage is not suitable for the feeding detection model.

[0048] In some disclosed embodiments, the optimization training module 44 includes a first return submodule for returning to the step of detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain a first feeding intensity after executing the step of optimizing the feeding detection model based on the sample audio set.

[0049] In some disclosed embodiments, the optimization training module 44 includes a second return submodule for returning to the step of detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model in response to the model analysis result characterizing that the current growth stage is adapted to the feeding detection model to obtain a first feeding intensity.

[0050] In some disclosed embodiments, the update analysis module 43 includes an audio selection submodule for selecting each feeding audio after the most recent optimization training of the feeding detection model as a historical audio; the update analysis module 43 includes a calibration statistics submodule for obtaining a calibration statistics result based on the calibration results of each historical audio regarding the feeding intensity; wherein the calibration result includes: the historical audio is a first audio in which the first feeding intensity is consistent with the second feeding intensity, or the historical audio is a second audio in which the first feeding intensity is inconsistent with the second feeding intensity, and the calibration statistics result includes at least one of the following: a first proportion of the first audio, a second proportion of the second audio; the update analysis module 43 includes a result analysis submodule for obtaining a model analysis result based on the calibration statistics result.

[0051] In some disclosed embodiments, the result analysis submodule includes a first acquisition unit for obtaining a model analysis result based on a size comparison result between the first proportion and a first threshold value when the verification statistical result includes a first proportion; the result analysis submodule includes a second acquisition unit for obtaining a model analysis result based on a size comparison result between the second proportion and a second threshold value when the verification statistical result includes a second proportion.

[0052] In some disclosed embodiments, the update analysis module 43 includes a sample deletion submodule for deleting at least part of the sample audio in the sample audio set before the most recent optimization training of the self-feeding detection model; the update analysis module 43 includes a sample retention submodule for retaining the sample audio in the sample audio set before the most recent optimization training of the self-feeding detection model.

[0053] In some disclosed embodiments, the eating detection module 41 includes a feature extraction submodule for extracting feature spectra of several feature types of eating audio; the eating detection module 41 includes a weight acquisition submodule for acquiring sub-weights of various feature spectra in the same frequency band; the eating detection module 41 includes a spectrogram weighting submodule for weighting the sub-spectra of various feature spectra in corresponding frequency bands based on the sub-weights of various feature spectra in the same frequency band to obtain weighted spectra; the eating detection module 41 includes an intensity prediction submodule for making predictions based on the weighted spectrogram to obtain a first eating intensity.

[0054] In some disclosed embodiments, the weight acquisition submodule includes an energy intensity acquisition unit, which is used to obtain the energy intensity of various characteristic spectra in the same frequency band; the weight acquisition submodule includes a characteristic spectrum selection unit, which is used to respectively select various characteristic spectra as the current spectrum, and select characteristic spectra other than the current spectrum as reference spectra; the weight acquisition submodule includes a frequency band weight calculation unit, which is used to obtain the sub-weight of the current spectrum in the corresponding frequency band based on the ratio of the energy intensity of the current spectrum and the reference spectrum in the same frequency band.

[0055] See also Figure 5 , Figure 5 : is a schematic diagram of the framework of an embodiment of an electronic device of the present application. The electronic device 50 at least includes a memory 51 and a processor 52 coupled to each other, the memory 51 at least stores program instructions, and the processor 52 is used to execute the program instructions to implement the steps in any of the above-mentioned embodiments of the method for detecting the intensity of food intake. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. As a possible example, the electronic device 50 may include but is not limited to a server, etc., and the specific type of the electronic device 50 is not limited here.

[0056] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps in any of the above-mentioned feeding intensity detection method embodiments. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 52 can be implemented by an integrated circuit chip.

[0057] In the above scheme, the electronic device 50 detects the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain a first feeding intensity, and the feeding detection model is trained based on the sample audio set, and then obtains a second feeding intensity after the first feeding intensity is verified, so as to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and obtains a model analysis result characterizing whether the current growth stage of the target aquatic product is suitable for the feeding detection model based on the verification statistics of the feeding intensity, and determines whether to perform the feeding detection based on the sample audio set based on the model analysis result. The feeding detection model is optimized and trained. Therefore, on the one hand, it is different from using computer vision technology and using audio data for feeding detection. It can avoid the influence of natural conditions such as whether the water is clear, whether the light is sufficient, and whether the background is stable as much as possible on feeding detection. It only needs to pay attention to the feeding sound, which is usually a stable sound of aquatic products during the feeding process. On the other hand, since the feeding verification is also statistically analyzed during the feeding detection process, and it is determined whether to update the training feeding detection model based on this, the applicability of the feeding detection model can be synchronized with the growth stage of aquatic products as much as possible. Therefore, the accuracy and stability of detecting feeding intensity can be improved, especially in deep-sea aquaculture scenarios.

[0058] See also Figure 6 , Figure 6 Schematic diagram of a framework of an embodiment of a computer-readable storage medium 60 of the present application. The computer-readable storage medium 60 stores program instructions 61 that can be executed by a processor, and the program instructions 61 are used to implement the steps in any of the above-mentioned food intake intensity detection method embodiments.

[0059] In the above scheme, the computer-readable storage medium 60 detects the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain a first feeding intensity, and the feeding detection model is obtained based on the training of the sample audio set, and then obtains the second feeding intensity after the first feeding intensity is verified, so as to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and obtains the model analysis result characterizing whether the current growth stage of the target aquatic product is suitable for the feeding detection model based on the statistical result of the feeding intensity, and determines whether the target aquatic product is suitable for the feeding detection model based on the sample audio based on the model analysis result. The feeding detection model is optimized and trained by the set. Therefore, on the one hand, it is different from the use of computer vision technology and the use of audio data for feeding detection. It can avoid the influence of natural conditions such as whether the water is clear, whether the light is sufficient, and whether the background is stable as much as possible on the feeding detection, and only needs to pay attention to the feeding sound, which is usually a stable existence of aquatic products during the feeding process. On the other hand, since the feeding verification is also statistically analyzed during the feeding detection process, and it is determined whether to update the training feeding detection model based on this, the applicability of the feeding detection model can be synchronized with the growth stage of aquatic products as much as possible. Therefore, the accuracy and stability of detecting feeding intensity can be improved, especially in deep-sea aquaculture scenarios.

[0060] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0061] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0062] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0063] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0064] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0065] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0066] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A method for detecting feeding intensity, characterized in that: include: Detecting the feeding audio of the target aquatic product in the target water area based on a feeding detection model to obtain a first feeding intensity; wherein the feeding detection model is trained based on a sample audio set; Obtaining a second feeding intensity after the first feeding intensity has been calibrated; Based on the feeding audio marked with the second feeding intensity and used as the sample audio, the sample audio set is updated, and based on the verification statistical result of the feeding intensity, a model analysis result characterizing whether the current growth stage of the target aquatic product is suitable for the feeding detection model is obtained; Based on the model analysis result, determine whether to optimize the food ingestion detection model based on the sample audio set.

2. The method according to claim 1, characterized in that The determining, based on the model analysis result, whether to optimize the food ingestion detection model based on the sample audio set comprises at least: In response to the model analysis result indicating that the current growth stage is not suitable for the feeding detection model, the feeding detection model is optimized and trained based on the sample audio set.

3. The method according to claim 1 or 2, characterized in that: After executing the step of optimizing and training the food ingestion detection model based on the sample audio set, the method further includes: Return to the step of detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain the first feeding intensity.

4. The method according to claim 1, characterized in that The determining, based on the model analysis result, whether to optimize the food ingestion detection model based on the sample audio set comprises at least: In response to the model analysis result indicating that the current growth stage is adapted to the feeding detection model, the step of detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model is returned to obtain a first feeding intensity.

5. The method according to claim 1, characterized in that: The statistical result of the feeding intensity-based calibration is used to obtain a model analysis result that characterizes whether the current growth stage of the target aquatic product is suitable for the feeding detection model, including: Select each of the food ingestion audios after the most recent optimization training of the food ingestion detection model as the historical audios; The verification statistical result is obtained based on the verification results of each of the historical audios regarding the feeding intensity; wherein the verification result includes: the historical audio is a first audio in which the first feeding intensity is consistent with the second feeding intensity, or the historical audio is a second audio in which the first feeding intensity is inconsistent with the second feeding intensity, and the verification statistical result includes at least one of the following: a first proportion of the first audio, a second proportion of the second audio; Based on the verification statistical results, the model analysis results are obtained.

6. The method according to claim 5, characterized in that The obtaining of the model analysis result based on the verification statistical result includes at least one of the following: In a case where the verification statistical result includes the first proportion, obtaining the model analysis result based on a comparison result between the first proportion and a first threshold value; In a case where the verification statistical result includes the second proportion, the model analysis result is obtained based on a comparison result between the second proportion and a second threshold value.

7. The method according to claim 1, characterized in that The updating of the sample audio set based on the food intake audio labeled with the second food intake intensity and serving as the sample audio includes any one of the following: Deleting at least part of the sample audio in the sample audio set before the most recent optimization training of the food ingestion detection model; The sample audios in the sample audio set are retained since the most recent optimization training of the food ingestion detection model.

8. The method according to claim 1, characterized in that: The method of detecting the feeding audio of the target aquatic product in the target water area based on the feeding detection model to obtain the first feeding intensity includes: Extracting characteristic spectrograms of several characteristic types of the food ingestion audio; Obtaining sub-weights of various characteristic spectra in the same frequency band; Based on the sub-weights of the various characteristic spectra in the same frequency band, weighting the sub-spectra in the corresponding frequency band of the various characteristic spectra to obtain a weighted spectra; Prediction is performed based on the weighted spectrogram to obtain the first feeding intensity.

9. The method according to claim 8, characterized in that The obtaining of sub-weights of various characteristic spectra in the same frequency band includes: Obtaining energy intensities of various characteristic spectra in the same frequency band; Selecting various characteristic spectra as current spectra respectively, and selecting characteristic spectra other than the current spectra as reference spectra; Based on the ratio of the energy intensities of the current spectrum and the reference spectrum in the same frequency band, the sub-weight of the current spectrum in the corresponding frequency band is obtained.

10. A food intake intensity detection device, characterized in that: include: A feeding detection module, used to detect the feeding audio of the target aquatic product in the target water area based on a feeding detection model to obtain a first feeding intensity; wherein the feeding detection model is obtained by training based on a sample audio set; An intensity checking module, used for obtaining a second food intake intensity after the first food intake intensity is checked; An updating analysis module is used to update the sample audio set based on the feeding audio marked with the second feeding intensity and used as the sample audio, and obtain a model analysis result characterizing whether the current growth stage of the target aquatic product is suitable for the feeding detection model based on the verification statistical result of the feeding intensity; The optimization training module is used to determine whether to optimize the food ingestion detection model based on the sample audio set based on the model analysis result.

11. An electronic device, characterized in that: The invention at least comprises a memory and a processor coupled to each other, wherein the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the method for detecting the food intake intensity according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the method for detecting the food intake intensity according to any one of claims 1 to 9.

Images