Piglet extrusion recognition method and device based on large language model

Abstract

The invention discloses a piglet extrusion recognition method and device based on a big language model, and relates to the field of animal behavior monitoring, and the method comprises the steps: obtaining a plurality of first audio sequences, and carrying out the matching of the plurality of first audio sequences with preset piglet extrusion sound features, and obtaining a matched audio sequence; wherein the plurality of first audio sequences are obtained based on an annular microphone array installed in a pig farm; acquiring a first video sequence acquired at an acquisition position corresponding to the matched audio sequence; and based on a large language model, in combination with a preset cue word and the first video sequence, obtaining an extrusion recognition result, and taking piglet rescue measures according to the extrusion recognition result, wherein the large language model is obtained by training based on acquired piglet extrusion image data. According to the embodiment of the invention, the accuracy of piglet extrusion recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of animal behavior monitoring, and in particular to a method and device for piglet squeeze recognition based on a large language model. Background Art

[0002] Newborn piglets are generally weak, and if the sow is not in the right position when feeding them, the piglets are easily crushed and killed. Therefore, it is necessary to monitor the crushing of piglets, identify the dangerous situation of piglet crushing and take measures to improve the survival rate of piglets.

[0003] The existing technologies mainly include piglet monitoring methods by counting the number of piglets, identifying piglet sounds, and identifying piglet sounds first and then combining them with video recognition, but all of them have defects. Piglets themselves are dead and their number is not stable. If the piglets are not completely squeezed, the number of piglets counted will remain unchanged. The method of determining whether the piglets are squeezed by counting the number of piglets is not reliable, and the accuracy is questionable. The pig farm pens are side by side and close to each other. It is generally difficult to determine the pens where squeezing occurs by piglet sound recognition. There may be missed recognition or misidentification, and the accuracy is not ideal. In the method of identifying piglet sounds first and then combining them with video recognition, there is a situation where the piglets in one pen make squeezing sounds while the sows in another pen are in a dangerous posture. This situation will lead to misidentification, incorrect positioning of the pen where the piglets are squeezed, and triggering measures for the wrong pen. Therefore, how to improve the accuracy of piglet squeezing recognition is still a problem that needs to be solved in the existing technology. Summary of the invention

[0004] The present application provides a method and device for piglet squeeze recognition based on a large language model to solve the technical problem of low accuracy in the recognition of piglet squeeze in the prior art.

[0005] According to a first aspect of the implementation scheme of the present application, a method for identifying piglet squeezing based on a large language model is provided, comprising:

[0006] Acquire multiple first audio sequences, and match the multiple first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences; wherein the multiple first audio sequences are obtained based on a circular microphone array installed in the pig farm;

[0007] Acquire a first video sequence captured at a capture position corresponding to the matching audio sequence;

[0008] Based on the large language model, combined with the preset prompt words and the first video sequence, a squeeze recognition result is obtained, so as to take piglet rescue measures according to the squeeze recognition result; wherein the large language model is trained based on the collected piglet squeeze image data.

[0009] The present application first obtains multiple first audio sequences through a circular microphone array in a pig farm, and then matches them with preset piglet squeezing sound features to obtain matching audio sequences. Through the correspondence between each microphone in the circular microphone array and the multiple first audio sequences, the audio sequence can be associated with the spatial position where the event occurred, so that the direction of the squeezing event can be determined by matching the piglet squeezing sound features, thereby improving the accuracy of piglet squeezing recognition; then, a first video sequence collected at a collection position corresponding to the matching audio sequence is obtained, and after determining the direction of the squeezing event through audio, video information can be obtained for further judgment, thereby avoiding incorrect positioning of the piglet squeezing pen and improving recognition accuracy, and the squeezing recognition result is obtained by deducing a large language model based on the piglet squeezing sound data. The large language model can improve the interpretability of the piglet squeezing image recognition task, thereby reducing the deviation in piglet squeezing image recognition and improving the accuracy of piglet squeezing recognition.

[0010] In some implementations of the present application, obtaining a plurality of first audio sequences specifically includes:

[0011] According to the annular microphone array, a central audio sequence and a plurality of initial audio sequences are obtained; wherein the central audio sequence is obtained by a microphone located at the center of the annular microphone array;

[0012] According to the central audio sequence, the multiple initial audio sequences are subjected to denoising and filtering to obtain multiple first audio sequences.

[0013] The present application first obtains a central audio sequence and multiple initial audio sequences through a circular microphone array, and then denoises and filters the multiple initial audio sequences according to the central audio sequence to obtain multiple first audio sequences. The multiple initial audio sequences can be corrected according to the central audio sequence, and the obtained multiple first audio sequences are associated with the array space arrangement of the circular microphone array, so that in subsequent matching, the direction of the squeezing event can be obtained through the association relationship combined with the matched audio sequence, thereby improving the accuracy of piglet squeezing recognition.

[0014] In certain embodiments of the present application, matching the plurality of first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences specifically includes:

[0015] Calculate the sound feature of the second audio sequence based on the Mel-frequency cepstral coefficient to obtain the first sound feature; wherein the second audio sequence is the one with the largest sound intensity among the multiple first audio sequences;

[0016] Calculating the similarity between the first sound feature and a preset piglet squeezing sound feature;

[0017] When the similarity exceeds a preset similarity threshold, the second audio sequence is used as a matching audio sequence.

[0018] The present application first calculates the first sound feature of the second audio sequence with the largest sound intensity among multiple first audio sequences based on the Mel-cephalogram coefficient, then calculates the similarity between the first sound feature and the preset piglet squeeze sound feature, and determines the second audio sequence as a matching audio sequence when the similarity exceeds a threshold. By screening the audio sequence by sound intensity, the direction of the sound source can be determined, thereby determining the direction of the squeeze event. Then, the sound feature is calculated and the matching audio sequence is determined by the similarity, and the occurrence of the piglet squeeze event can be further determined by the piglet squeeze sound feature, thereby improving the accuracy of piglet squeeze recognition.

[0019] In certain embodiments of the present application, obtaining the first video sequence acquired at the acquisition position corresponding to the matching audio sequence specifically includes:

[0020] Acquire an initial video sequence acquired at a capture position corresponding to the matching audio sequence;

[0021] According to the start timestamp of the matching audio sequence and the preset acquisition duration, the initial video sequence is cropped to obtain an acquired video sequence;

[0022] The acquired video sequence is frame sampled according to a preset frame sampling rate to obtain a first video sequence.

[0023] The present application first obtains an initial video sequence collected at a collection position corresponding to a matching audio sequence, and after determining the direction of the squeezing event through audio, obtains a video collected at a collection position corresponding to the matching audio sequence to further judge the piglet squeezing event, thereby avoiding incorrect judgment of the piglet squeezing pen caused by sound, and improving recognition accuracy. The initial video sequence is then cropped according to the starting timestamp of the matching audio sequence and the preset collection time, and frame sampling is performed to obtain a first video sequence. Based on the actual current piglet squeezing recognition task, the data amount and data format required for the task can be adjusted to better meet the actual needs of the task.

[0024] In certain embodiments of the present application, the extrusion recognition result is obtained based on the large language model in combination with the preset prompt word and the first video sequence, specifically including:

[0025] Encoding the first video sequence to obtain a first encoded text;

[0026] The preset prompt word and the first encoded text are input into the large language model to obtain a squeeze recognition result.

[0027] The present application first encodes the first video sequence to obtain a first encoded text, and can adjust the first video sequence to a data format that conforms to the input of a large language model based on the current actual piglet squeeze recognition task, which is more in line with the actual needs of the task, and then inputs the preset prompt words and the first encoded text into the large language model to obtain a squeeze recognition result. The large language model can improve the interpretability of the piglet squeeze recognition task, thereby reducing the deviation in piglet squeeze image recognition and improving the accuracy of piglet squeeze recognition.

[0028] According to a second aspect of the implementation of the present application, there is provided a piglet squeeze identification device based on a large language model, comprising an audio matching module, a video acquisition module and a squeeze identification module;

[0029] The audio matching module is used to obtain a plurality of first audio sequences, and match the plurality of first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences; wherein the plurality of first audio sequences are obtained based on a circular microphone array installed in the pig farm;

[0030] The video acquisition module is used to acquire a first video sequence acquired at a capture position corresponding to the matching audio sequence;

[0031] The squeeze recognition module is used to obtain a squeeze recognition result based on a large language model in combination with preset prompt words and the first video sequence, so as to take piglet rescue measures according to the squeeze recognition result; wherein the large language model is trained based on the collected piglet squeeze image data.

[0032] In some embodiments of the present application, the audio matching module includes an audio acquisition unit; the audio acquisition unit includes an audio acquisition subunit and an audio denoising subunit;

[0033] The audio acquisition subunit is used to acquire a central audio sequence and a plurality of initial audio sequences according to the annular microphone array; wherein the central audio sequence is acquired by a microphone located at the center of the annular microphone array;

[0034] The audio denoising subunit is used to perform denoising filtering on the multiple initial audio sequences according to the central audio sequence to obtain multiple first audio sequences.

[0035] In some embodiments of the present application, the audio matching module includes an audio matching unit; the audio matching unit includes a feature calculation subunit, a similarity calculation subunit and an audio matching subunit;

[0036] The feature calculation subunit is used to calculate the sound feature of the second audio sequence based on the Mel-frequency cepstral coefficient to obtain the first sound feature; wherein the second audio sequence is the one with the largest sound intensity among the multiple first audio sequences;

[0037] The similarity calculation subunit is used to calculate the similarity between the first sound feature and the preset piglet squeezing sound feature;

[0038] The audio matching subunit is configured to use the second audio sequence as a matching audio sequence when the similarity exceeds a preset similarity threshold.

[0039] In certain embodiments of the present application, the video acquisition module includes a video acquisition unit, a video cropping unit, and a video sampling unit;

[0040] The video acquisition unit is used to acquire an initial video sequence acquired at a acquisition position corresponding to the matching audio sequence;

[0041] The video cropping unit is used to crop the initial video sequence according to the start timestamp of the matching audio sequence and the preset acquisition duration to obtain an acquired video sequence;

[0042] The video sampling unit is used to perform frame sampling on the acquired video sequence according to a preset frame sampling rate to obtain a first video sequence.

[0043] In certain embodiments of the present application, the squeeze identification module includes a video encoding unit and a squeeze identification unit;

[0044] The video encoding unit is used to encode the first video sequence to obtain a first encoded text;

[0045] The extrusion recognition unit is used to input a preset prompt word and the first encoded text into the large language model to obtain an extrusion recognition result.

[0046] The present application first obtains multiple first audio sequences through a circular microphone array in a pig farm, and then matches them with preset piglet squeezing sound features to obtain matching audio sequences. Through the correspondence between each microphone in the circular microphone array and the multiple first audio sequences, the audio sequence can be associated with the spatial position where the event occurred, so that the direction of the squeezing event can be determined by matching the piglet squeezing sound features, thereby improving the accuracy of piglet squeezing recognition; then, a first video sequence collected at a collection position corresponding to the matching audio sequence is obtained, and after determining the direction of the squeezing event through audio, video information can be obtained for further judgment, thereby avoiding incorrect positioning of the piglet squeezing pen and improving recognition accuracy, and the squeezing recognition result is obtained by deducing a large language model based on the piglet squeezing sound data. The large language model can improve the interpretability of the piglet squeezing image recognition task, thereby reducing the deviation in piglet squeezing image recognition and improving the accuracy of piglet squeezing recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 : A schematic diagram of a process flow of a piglet squeeze recognition method based on a large language model shown in certain embodiments of the present application;

[0048] Figure 2 : A module structure diagram of a piglet squeeze recognition device based on a large language model shown in certain embodiments of the present application;

[0049] Figure 3 : A schematic diagram of the arrangement of a circular microphone array shown in certain embodiments of the present application. DETAILED DESCRIPTION

[0050] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of the present application, and should not be construed as limitations on the embodiments of the present application. Based on the embodiments shown in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0051] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, unless otherwise clearly and specifically defined, "multiple" and "several" mean two or more.

[0052] Among the existing methods for piglet squeeze identification and monitoring, the method of counting the number of piglets is unreliable because of the certain mortality rate of piglets themselves and the actual defects that the number of piglets counted remains unchanged when they are not completely squeezed; the method of using piglet sound recognition is difficult to accurately identify the squeezing pen when the pig pens are side by side and close to each other in the pig farm, and there are cases of missed recognition or misidentification, and the accuracy is not ideal; the method of first identifying the piglet sound and then combining it with video recognition has the possibility that when the piglets in one pen make a squeezing sound, the sows in another pen are in a dangerous posture, which will lead to the wrong location of the squeezing pen of the piglets and trigger measures for the wrong pen. The above methods all have certain defects. Therefore, how to improve the accuracy of piglet squeezing recognition is still a problem that needs to be solved urgently in the existing technology.

[0053] Based on the above technical background, please refer to Figure 1 The embodiment of the present application provides a method for identifying piglet squeezing based on a large language model, including steps S101 to S103, each of which is as follows:

[0054] Step S101: Acquire multiple first audio sequences, and match the multiple first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences; wherein the multiple first audio sequences are obtained based on a circular microphone array installed in a pig farm.

[0055] In certain embodiments of the present application, the annular microphone array includes a microphone at the center of the array and a plurality of microphones arranged in an annular direction and facing outward; the microphone at the center of the array is oriented vertically upward or downward to the ground; the tail of each microphone arranged in an annular direction points to the microphone at the center of the array, the head of each microphone points to the outside of the annular direction, and the head and tail of each microphone are on the same axis as the center of the array. For example, see Figure 3 , is a schematic diagram of the arrangement of a circular microphone array, wherein mic0 is the microphone at the center of the array; the number of microphones arranged in a circular shape is 8, specifically mic1-mic8, and their relative rotation angles are all 45 degrees.

[0056] In certain embodiments of the present application, in order to improve positioning accuracy, the annular microphone array can be set to one or more; when set to multiple, after each array obtains the corresponding matching audio sequence, a positioning point is obtained through triangulation based on the matching audio sequence corresponding to each array.

[0057] In some implementations of the present application, obtaining a plurality of first audio sequences specifically includes:

[0058] According to the annular microphone array, a central audio sequence and a plurality of initial audio sequences are obtained; wherein the central audio sequence is obtained by a microphone located at the center of the annular microphone array;

[0059] According to the central audio sequence, the multiple initial audio sequences are subjected to denoising and filtering to obtain multiple first audio sequences.

[0060] In some implementations of the present application, the denoising and filtering of the multiple initial audio sequences is performed according to the central audio sequence to obtain multiple first audio sequences, specifically:

[0061] Performing subtraction and filtering on the multiple initial audio sequences and the central audio sequence respectively to obtain multiple processed audio sequences;

[0062] De-noising and filtering are performed on the multiple processed audio sequences to obtain multiple first audio sequences.

[0063] The present application first obtains a central audio sequence and multiple initial audio sequences through a circular microphone array, and then denoises and filters the multiple initial audio sequences according to the central audio sequence to obtain multiple first audio sequences. The multiple initial audio sequences can be corrected according to the central audio sequence, and the obtained multiple first audio sequences are associated with the array space arrangement of the circular microphone array, so that in subsequent matching, the direction of the squeezing event can be obtained through the association relationship combined with the matched audio sequence, thereby improving the accuracy of piglet squeezing recognition.

[0064] In certain embodiments of the present application, matching the plurality of first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences specifically includes:

[0065] Calculate the sound feature of the second audio sequence based on the Mel-frequency cepstral coefficient to obtain the first sound feature; wherein the second audio sequence is the one with the largest sound intensity among the multiple first audio sequences;

[0066] Calculating the similarity between the first sound feature and a preset piglet squeezing sound feature;

[0067] When the similarity exceeds a preset similarity threshold, the second audio sequence is used as a matching audio sequence.

[0068] In certain embodiments of the present application, the methods for calculating the similarity between the first sound feature and the preset piglet squeezing sound feature include but are not limited to Mel-spectrum comparison, Chroma feature comparison, waveform comparison and cosine similarity comparison, and the preferred implementation is cosine similarity comparison.

[0069] In certain embodiments of the present application, the preferred value of the preset similarity threshold is 0.8.

[0070] The present application first calculates the first sound feature of the second audio sequence with the largest sound intensity among multiple first audio sequences based on the Mel-cephalogram coefficient, then calculates the similarity between the first sound feature and the preset piglet squeeze sound feature, and determines the second audio sequence as a matching audio sequence when the similarity exceeds a threshold. By screening the audio sequence by sound intensity, the direction of the sound source can be determined, thereby determining the direction of the squeeze event. Then, the sound feature is calculated and the matching audio sequence is determined by the similarity, and the occurrence of the piglet squeeze event can be further determined by the piglet squeeze sound feature, thereby improving the accuracy of piglet squeeze recognition.

[0071] Step S102: Acquire a first video sequence captured at a capture position corresponding to the matching audio sequence.

[0072] In certain embodiments of the present application, obtaining the first video sequence acquired at the acquisition position corresponding to the matching audio sequence specifically includes:

[0073] Acquire an initial video sequence acquired at a capture position corresponding to the matching audio sequence;

[0074] According to the start timestamp of the matching audio sequence and the preset acquisition duration, the initial video sequence is cropped to obtain an acquired video sequence;

[0075] The acquired video sequence is frame sampled according to a preset frame sampling rate to obtain a first video sequence.

[0076] In certain embodiments of the present application, the preferred value of the preset acquisition time length is 30s; the preferred value range of the frame sampling rate is 1 / 80 to 1 / 50, and the preferred value is 1 / 50, that is, 1 frame is sampled every 50 frames.

[0077] The present application first obtains an initial video sequence collected at a collection position corresponding to a matching audio sequence, and after determining the direction of the squeezing event through audio, obtains a video collected at a collection position corresponding to the matching audio sequence to further judge the piglet squeezing event, thereby avoiding incorrect judgment of the piglet squeezing pen caused by sound, and improving recognition accuracy. The initial video sequence is then cropped according to the starting timestamp of the matching audio sequence and the preset collection time, and frame sampling is performed to obtain a first video sequence. Based on the actual current piglet squeezing recognition task, the data amount and data format required for the task can be adjusted to better meet the actual needs of the task.

[0078] Step S103: Based on the large language model, combined with the preset prompt words and the first video sequence, a squeeze recognition result is obtained, so as to take piglet rescue measures according to the squeeze recognition result; wherein the large language model is trained based on the collected piglet squeeze image data.

[0079] In certain embodiments of the present application, the extrusion recognition result is obtained based on the large language model in combination with the preset prompt word and the first video sequence, specifically including:

[0080] Encoding the first video sequence to obtain a first encoded text;

[0081] The preset prompt word and the first encoded text are input into the large language model to obtain a squeeze recognition result.

[0082] In some embodiments of the present application, the preferred encoding method for encoding the first video sequence is Base64 encoding.

[0083] The present application first encodes the first video sequence to obtain a first encoded text, and can adjust the first video sequence to a data format that conforms to the input of a large language model based on the current actual piglet squeeze recognition task, which is more in line with the actual needs of the task, and then inputs the preset prompt words and the first encoded text into the large language model to obtain a squeeze recognition result. The large language model can improve the interpretability of the piglet squeeze recognition task, thereby reducing the deviation in piglet squeeze image recognition and improving the accuracy of piglet squeeze recognition.

[0084] Compared with the prior art, the present application first obtains multiple first audio sequences through a circular microphone array in a pig farm, and then matches them with preset piglet squeezing sound features to obtain matching audio sequences. Through the correspondence between each microphone in the circular microphone array and the multiple first audio sequences, the audio sequence can be associated with the spatial position of the event, so that the direction of the squeezing event can be determined by matching the piglet squeezing sound features, thereby improving the accuracy of piglet squeezing recognition; then, the first video sequence collected at the collection position corresponding to the matching audio sequence is obtained, and after determining the direction of the squeezing event through audio, video information can be obtained for further judgment, thereby avoiding incorrect positioning of the piglet squeezing pen and improving recognition accuracy, and the squeezing recognition result is derived through a large language model based on the piglet squeezing sound data. The large language model can improve the interpretability of the piglet squeezing image recognition task, thereby reducing the deviation in piglet squeezing image recognition and improving the accuracy of piglet squeezing recognition.

[0085] Corresponding to the previous method, see Figure 2 , the embodiment of the present application provides a piglet squeeze recognition device based on a large language model, including an audio matching module 210, a video acquisition module 220 and a squeeze recognition module 230;

[0086] The audio matching module 210 is used to obtain a plurality of first audio sequences, and match the plurality of first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences; wherein the plurality of first audio sequences are obtained based on a circular microphone array installed in the pig farm;

[0087] The video acquisition module 220 is used to acquire a first video sequence acquired at a capture position corresponding to the matching audio sequence;

[0088] The squeeze recognition module 230 is used to obtain a squeeze recognition result based on a large language model in combination with preset prompt words and the first video sequence, so as to take piglet rescue measures according to the squeeze recognition result; wherein the large language model is trained based on the collected piglet squeeze image data.

[0089] In some embodiments of the present application, the audio matching module 210 includes an audio acquisition unit; the audio acquisition unit includes an audio acquisition subunit and an audio denoising subunit;

[0090] The audio acquisition subunit is used to acquire a central audio sequence and a plurality of initial audio sequences according to the annular microphone array; wherein the central audio sequence is acquired by a microphone located at the center of the annular microphone array;

[0091] The audio denoising subunit is used to perform denoising filtering on the multiple initial audio sequences according to the central audio sequence to obtain multiple first audio sequences.

[0092] In some embodiments of the present application, the audio matching module 210 includes an audio matching unit; the audio matching unit includes a feature calculation subunit, a similarity calculation subunit and an audio matching subunit;

[0093] The feature calculation subunit is used to calculate the sound feature of the second audio sequence based on the Mel-frequency cepstral coefficient to obtain the first sound feature; wherein the second audio sequence is the one with the largest sound intensity among the multiple first audio sequences;

[0094] The similarity calculation subunit is used to calculate the similarity between the first sound feature and the preset piglet squeezing sound feature;

[0095] The audio matching subunit is configured to use the second audio sequence as a matching audio sequence when the similarity exceeds a preset similarity threshold.

[0096] In some embodiments of the present application, the video acquisition module 220 includes a video acquisition unit, a video cropping unit, and a video sampling unit;

[0097] The video acquisition unit is used to acquire an initial video sequence acquired at a acquisition position corresponding to the matching audio sequence;

[0098] The video cropping unit is used to crop the initial video sequence according to the start timestamp of the matching audio sequence and the preset acquisition duration to obtain an acquired video sequence;

[0099] The video sampling unit is used to perform frame sampling on the acquired video sequence according to a preset frame sampling rate to obtain a first video sequence.

[0100] In some embodiments of the present application, the squeeze identification module 230 includes a video encoding unit and a squeeze identification unit;

[0101] The video encoding unit is used to encode the first video sequence to obtain a first encoded text;

[0102] The extrusion recognition unit is used to input a preset prompt word and the first encoded text into the large language model to obtain an extrusion recognition result.

[0103] The present application first obtains multiple first audio sequences through a circular microphone array in a pig farm, and then matches them with preset piglet squeezing sound features to obtain matching audio sequences. Through the correspondence between each microphone in the circular microphone array and the multiple first audio sequences, the audio sequence can be associated with the spatial position where the event occurred, so that the direction of the squeezing event can be determined by matching the piglet squeezing sound features, thereby improving the accuracy of piglet squeezing recognition; then, a first video sequence collected at a collection position corresponding to the matching audio sequence is obtained, and after determining the direction of the squeezing event through audio, video information can be obtained for further judgment, thereby avoiding incorrect positioning of the piglet squeezing pen and improving recognition accuracy, and the squeezing recognition result is obtained by deducing a large language model based on the piglet squeezing sound data. The large language model can improve the interpretability of the piglet squeezing image recognition task, thereby reducing the deviation in piglet squeezing image recognition and improving the accuracy of piglet squeezing recognition.

[0104] It should be understood that the device provided in the embodiment of the present application corresponds to the aforementioned method, and a piglet squeeze identification device based on a large language model provided in the embodiment of the present application can implement a piglet squeeze identification method based on a large language model provided in any embodiment of the present application.

[0105] Adaptively, the embodiments of the present application also provide a computer device and a computer-readable storage medium.

[0106] The computer device comprises: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor;

[0107] Wherein, when the processor executes the computer program, a piglet squeezing recognition method based on a large language model of the present application is implemented.

[0108] The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute a piglet squeeze recognition method based on a large language model of the present application.

[0109] The above are some implementation methods of the present application, and the purpose, technical scheme and beneficial effects of the present application are further described in detail. It should be clear that the above-mentioned partial implementation methods of the present application cannot be understood as limiting the present application. It is particularly pointed out that for those skilled in the art, any changes, modifications, equivalent substitutions and variations made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for piglet squeeze recognition based on a large language model, characterized in that: include: Acquire multiple first audio sequences, and match the multiple first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences; wherein the multiple first audio sequences are obtained based on a circular microphone array installed in the pig farm; Acquire a first video sequence captured at a capture position corresponding to the matching audio sequence; Based on the large language model, combined with the preset prompt words and the first video sequence, a squeeze recognition result is obtained, so as to take piglet rescue measures according to the squeeze recognition result; wherein the large language model is trained based on the collected piglet squeeze image data.

2. A method for piglet squeeze recognition based on a large language model according to claim 1, characterized in that: The acquiring of the plurality of first audio sequences specifically includes: According to the annular microphone array, a central audio sequence and a plurality of initial audio sequences are obtained; wherein the central audio sequence is obtained by a microphone located at the center of the annular microphone array; According to the central audio sequence, the multiple initial audio sequences are subjected to denoising and filtering to obtain multiple first audio sequences.

3. The method for piglet squeeze recognition based on a large language model according to claim 1, characterized in that: The step of matching the plurality of first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences specifically includes: Calculate the sound feature of the second audio sequence based on the Mel-frequency cepstral coefficient to obtain the first sound feature; wherein the second audio sequence is the one with the largest sound intensity among the multiple first audio sequences; Calculating the similarity between the first sound feature and a preset piglet squeezing sound feature; When the similarity exceeds a preset similarity threshold, the second audio sequence is used as a matching audio sequence.

4. The method for piglet squeeze recognition based on a large language model according to claim 1, characterized in that: The step of obtaining a first video sequence acquired at an acquisition position corresponding to the matching audio sequence specifically includes: Acquire an initial video sequence acquired at a capture position corresponding to the matching audio sequence; According to the start timestamp of the matching audio sequence and the preset acquisition duration, the initial video sequence is cropped to obtain an acquired video sequence; The acquired video sequence is frame sampled according to a preset frame sampling rate to obtain a first video sequence.

5. The method for piglet squeeze recognition based on a large language model according to claim 1, characterized in that: The step of obtaining the squeeze recognition result based on the large language model and combining the preset prompt words and the first video sequence specifically includes: Encoding the first video sequence to obtain a first encoded text; The preset prompt word and the first encoded text are input into the large language model to obtain a squeeze recognition result.

6. A piglet squeeze recognition device based on a large language model, characterized in that: It includes an audio matching module, a video acquisition module and an extrusion recognition module; The audio matching module is used to obtain a plurality of first audio sequences, and match the plurality of first audio sequences with preset piglet squeezing sound features to obtain matching audio sequences; wherein the plurality of first audio sequences are obtained based on a circular microphone array installed in the pig farm; The video acquisition module is used to acquire a first video sequence acquired at a capture position corresponding to the matching audio sequence; The squeeze recognition module is used to obtain a squeeze recognition result based on a large language model in combination with preset prompt words and the first video sequence, so as to take piglet rescue measures according to the squeeze recognition result; wherein the large language model is trained based on the collected piglet squeeze image data.

7. A piglet squeeze recognition device based on a large language model according to claim 6, characterized in that: The audio matching module includes an audio acquisition unit; the audio acquisition unit includes an audio acquisition subunit and an audio denoising subunit; The audio acquisition subunit is used to acquire a central audio sequence and a plurality of initial audio sequences according to the annular microphone array; wherein the central audio sequence is acquired by a microphone located at the center of the annular microphone array; The audio denoising subunit is used to perform denoising filtering on the multiple initial audio sequences according to the central audio sequence to obtain multiple first audio sequences.

8. The piglet squeeze recognition device based on a large language model according to claim 6, characterized in that: The audio matching module includes an audio matching unit; the audio matching unit includes a feature calculation subunit, a similarity calculation subunit and an audio matching subunit; The feature calculation subunit is used to calculate the sound feature of the second audio sequence based on the Mel-frequency cepstral coefficient to obtain the first sound feature; wherein the second audio sequence is the one with the largest sound intensity among the multiple first audio sequences; The similarity calculation subunit is used to calculate the similarity between the first sound feature and the preset piglet squeezing sound feature; The audio matching subunit is configured to use the second audio sequence as a matching audio sequence when the similarity exceeds a preset similarity threshold.

9. The piglet squeeze recognition device based on a large language model according to claim 6, characterized in that: The video acquisition module includes a video acquisition unit, a video cropping unit and a video sampling unit; The video acquisition unit is used to acquire an initial video sequence acquired at a acquisition position corresponding to the matching audio sequence; The video cropping unit is used to crop the initial video sequence according to the start timestamp of the matching audio sequence and the preset acquisition duration to obtain an acquired video sequence; The video sampling unit is used to perform frame sampling on the acquired video sequence according to a preset frame sampling rate to obtain a first video sequence.

10. The piglet squeeze recognition device based on a large language model according to claim 6, characterized in that: The squeeze recognition module includes a video encoding unit and a squeeze recognition unit; The video encoding unit is used to encode the first video sequence to obtain a first encoded text; The extrusion recognition unit is used to input a preset prompt word and the first encoded text into the large language model to obtain an extrusion recognition result.

Images