Visual monitoring method for full-life-cycle growth state of toad breeding

Abstract

The invention discloses a full-life-cycle growth state visual monitoring method for toad breeding, and relates to the technical field of image matching. The method comprises the following steps: performing visual monitoring on a target toad to obtain a first toad contour image of the target toad; under the condition that monitoring of the target toad is interrupted, predicting a suspected position area of the target toad according to each first toad contour image; according to each first toad contour image and the region feature image of the suspected position region, determining a feature matching degree between each candidate toad and the target toad in the suspected position region; according to the second toad contour image of each reference toad in the toad group to which the target toad belongs, determining the growth trajectory conformity between each candidate toad and the target toad in the suspected position area; according to the feature matching degree and the growth trajectory conformity, determining the overall matching degree of each candidate toad and the target toad; and determining the candidate toad with the maximum overall matching degree as a target toad, and recovering visual monitoring of the target toad.

Description

Technical Field

[0001] The present invention relates to the technical field of image matching, and particularly relates to a visual monitoring method for the growth state of toads throughout their life cycle in farming. Background Art

[0002] Toad farming refers to the artificial breeding, reproduction, and management of toads. By monitoring the growth state of toads throughout their complete life cycle from fertilized eggs, tadpoles, metamorphosis, juveniles to adults until reproduction, including aspects such as body size, weight, appearance characteristics, and behavioral activities, the individual health and development level of toads can be accurately measured.

[0003] Currently, automated visual detection technology is used to monitor the growth state of toads throughout their life cycle in real time, and feature information is extracted through image processing methods to determine the growth stage, behavioral state, and possible health abnormalities of toads.

[0004] However, occlusion is likely to occur during the monitoring of toads, resulting in the inability to track toads completely and poor accuracy in monitoring the growth state of toads. Summary of the Invention

[0005] The embodiments of the present invention provide a visual monitoring method for the growth state of toads throughout their life cycle in farming, which can improve the accuracy of monitoring the growth state of toads.

[0006] In the first aspect of the embodiments of the present invention, a visual monitoring method for the growth state of toads throughout their life cycle in farming is provided, including:

[0007] Conduct visual monitoring on a target toad to obtain a first toad contour image of the target toad in real time;

[0008] In the case of interruption of monitoring the target toad, predict the suspected position area of the target toad according to each first toad contour image;

[0009] According to each first toad contour image and the regional feature image of the suspected position area, determine the feature matching degree between each candidate toad and the target toad in the suspected position area;

[0010] According to the second toad contour images of each reference toad in the toad group to which the target toad belongs, determine the growth trajectory conformity degree between each candidate toad and the target toad in the suspected position area. The toad group includes toads with similar growth trajectories, and the reference toad is a toad that has been normally monitored except the target toad in the toad group;

[0011] According to the feature matching degree and the growth trajectory conformity degree, determine the overall matching degree between each candidate toad and the target toad;

[0012] Determine the target toad with the largest overall matching degree, and resume visual monitoring of the target toad.

[0013] In the visual monitoring method for the growth state of the whole life cycle of toad breeding provided by the embodiment of the present invention, when the monitoring of the target toad is interrupted, first predict the suspected position area of the target toad according to each first toad contour image of the target toad. In this way, it can provide a general position for the subsequent positioning of the target toad and narrow the matching range. Then, determine the feature matching degree between each candidate toad and the target toad in the suspected position area according to each first toad contour image and the regional feature image of the suspected position area; at the same time, determine the growth trajectory compliance degree between each candidate toad and the target toad in the suspected position area according to the second toad contour images of each reference toad in the toad group to which the target toad belongs. In this way, relying on the reference toads with a growth trajectory similar to that of the target toad and the feature matching situation between each candidate toad and the target toad in the suspected position area, the position of the target toad in the suspected position area can be accurately determined. To sum up, in the process of toad monitoring, when the monitoring of the target toad is interrupted due to occlusion or other phenomena, the target toad can also be accurately tracked, thereby improving the accuracy of toad growth state monitoring. Brief Description of the Drawings

[0014] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is a schematic flowchart of the first visual monitoring method for the growth state of the whole life cycle of toad breeding provided by an embodiment of the present invention;

[0016] Figure 2 It is a schematic flowchart of the second visual monitoring method for the growth state of the whole life cycle of toad breeding provided by an embodiment of the present invention;

[0017] Figure 3 It is a schematic flowchart of the third visual monitoring method for the growth state of the whole life cycle of toad breeding provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0018] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details a visual monitoring method for the growth state of toads throughout their entire life cycle according to the present invention, including its specific implementation manners, structures, features, and effects, as follows. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this invention belongs.

[0020] It should be noted that in the technical solution of the present invention, the acquisition, storage, use, processing, etc. of data all comply with the relevant regulations of laws and regulations.

[0021] It should be noted that in the embodiments of the present invention, some existing industry solutions such as certain software, components, models, etc. may be mentioned. They should be regarded as exemplary, and their purpose is only to illustrate the feasibility in the implementation of the technical solution of the present invention, but it does not mean that the applicant has already or necessarily used this solution.

[0022] Toad farming refers to the artificial rearing, breeding, and management of toads. By monitoring the growth state of toads throughout their entire life cycle from fertilized eggs, tadpoles, metamorphosis periods, juveniles to adults and until reproduction, including aspects such as body size, weight, appearance characteristics, and behavioral activities, the individual health and development level of toads can be accurately measured.

[0023] Currently, automated visual inspection technology is used to monitor the growth state of toads throughout their entire life cycle in real time. Feature information is extracted through image processing methods to determine the growth stage, behavioral state, and possible health abnormalities of toads. However, during the process of toad monitoring, occlusion phenomena are likely to occur, resulting in the inability to track toads completely, and thus the accuracy of toad growth state monitoring is relatively poor.

[0024] The purpose of the present invention is to provide a visual monitoring method for the growth state of toads throughout their life cycle. In the visual monitoring method for the growth state of toads throughout their life cycle provided by the embodiments of the present invention, in the case where the monitoring of the target toad is interrupted, first, according to each first toad contour image of the target toad, the suspected position area of the target toad is predicted. In this way, it is possible to provide a rough position for the subsequent positioning of the target toad and narrow the matching range. Then, according to each first toad contour image and the regional feature image of the suspected position area, the feature matching degree between each candidate toad and the target toad in the suspected position area is determined; at the same time, according to the second toad contour images of each reference toad in the toad group to which the target toad belongs, the growth trajectory compliance degree between each candidate toad and the target toad in the suspected position area is determined. In this way, relying on the reference toads with a growth trajectory similar to that of the target toad and the feature matching situation between each candidate toad and the target toad in the suspected position area, the position of the target toad in the suspected position area can be accurately determined. To sum up, in the process of toad monitoring, in the case where the monitoring of the target toad is interrupted due to occlusion or other phenomena, the target toad can also be accurately tracked, thereby improving the accuracy of toad growth state monitoring.

[0025] The following introduces a specific embodiment of a visual monitoring method for the growth state of toads throughout their life cycle provided by the embodiments of the present invention.

[0026] Figure 1 A flowchart of a visual monitoring method for the growth state of toads throughout their life cycle is provided. This visual monitoring method for the growth state of toads throughout their life cycle can be applied to a server, and this visual monitoring method for the growth state of toads throughout their life cycle can include the following S101 to S106.

[0027] S101, conduct visual monitoring on the target toad and obtain the first toad contour image of the target toad in real time.

[0028] In this embodiment, the target toad is used to represent the toad that needs to be visually monitored.

[0029] As an example, install monitoring devices such as high-definition cameras or other visual sensors in the activity area of the target toad to ensure that clear images can be captured. Then, the server collects target toad images through the monitoring device at a preset frequency, and uses image processing algorithms (such as edge detection, contour extraction, etc.) to process the collected target toad images and extract the first toad contour image of the target toad.

[0030] S102, in the case where the monitoring of the target toad is interrupted, predict the suspected position area of the target toad according to each first toad contour image.

[0031] In this embodiment, the interruption of monitoring the target toad means losing the position of the target toad. For example, when there is an obstacle blocking the target toad, the situation of interrupting the monitoring of the target toad will occur.

[0032] The suspected position area is used to represent the area where the target toad is suspected to be located after the interruption of monitoring the target toad, based on speculation.

[0033] As an example, when the server cannot collect the first toad contour image of the toad through the monitoring device, it can be considered at this time that the position of the target toad is lost, that is, the monitoring of the target toad is interrupted.

[0034] At this time, the server uses the historical first toad contour image of the target toad, and through methods such as machine learning or statistical analysis, speculates the movement law of the target toad, so as to predict the suspected position area that the target toad will reach after being blocked.

[0035] S103. According to each first toad contour image and the regional feature image of the suspected position area, determine the feature matching degree between each candidate toad and the target toad in the suspected position area.

[0036] In this embodiment, the regional feature image is used to represent the image of the suspected position area at the current moment, and the candidate toad is used to represent each toad included in the regional feature image.

[0037] The feature matching degree is used to represent the similarity degree of features between the candidate toad and the target toad. For example, it can include the similarity degree of features such as skin roughness, spot distribution, body edge, eyes, and limb contour.

[0038] As an example, the server identifies all candidate toads from the regional feature image of the suspected position area, and extracts the features of the candidate toads and the target toad respectively according to the first toad contour image of the target toad and the regional feature image of the suspected position area.

[0039] Then, use a feature matching algorithm (such as template matching, similarity calculation, etc.) to calculate the feature matching degree between each candidate toad and the target toad.

[0040] S104. According to the second toad contour images of each reference toad in the toad group to which the target toad belongs, determine the growth trajectory compliance degree between each candidate toad and the target toad in the suspected position area. The toad group includes each toad with similar growth trajectories, and the reference toad is the toad that is normally monitored except the target toad in the toad group.

[0041] In this embodiment, the toad group includes multiple toads, and the growth trajectories of the toads in the toad group are similar. The reference toad is the toad in the toad group that is normally monitored except for the target toad, that is, the toad in the toad group without monitoring interruption.

[0042] The second toad contour image is used to represent the toad contour image corresponding to the reference toad, and the growth trajectory compliance is used to represent the degree of compliance of the growth trajectory between the candidate toad and the target toad.

[0043] As an example, the server obtains the second toad contour images of each reference toad in the toad group to which the target toad belongs. Then, based on the second toad contour images of the reference toads, using technologies such as image processing and time series analysis, the growth trajectory of the reference toads is analyzed.

[0044] Then, based on the growth trajectories of the reference toads, the current growth status of the target toad is inferred. Finally, the current growth status of the candidate toad is compared with the inferred current growth status of the target toad to determine the growth trajectory compliance between the candidate toad and the target toad.

[0045] S105. Determine the overall matching degree between each candidate toad and the target toad according to the feature matching degree and the growth trajectory compliance.

[0046] In this embodiment, different weights are assigned to the feature matching degree and the growth trajectory compliance according to actual needs.

[0047] Then, the server combines the feature matching degree and the growth trajectory compliance using weighted average or other comprehensive calculation methods to calculate the overall matching degree between each candidate toad and the target toad.

[0048] As an example, the overall matching degree between the candidate toad and the target toad can be specifically determined by the following formula 1:

[0049] Y i =Z i ×(1+F i ) Formula 1

[0050] In Formula 1, Y i is used to represent the overall matching degree between the i-th candidate toad and the target toad, Z i is used to represent the feature matching degree between the i-th candidate toad and the target toad, and F i is used to represent the growth trajectory compliance between the i-th candidate toad and the target toad.

[0051] S106. Determine the candidate toad with the largest overall matching degree as the target toad and resume visual monitoring of the target toad.

[0052] In this embodiment, among all candidate toads, the toad with the largest overall matching degree is determined as the target toad.

[0053] Then, the server controls and adjusts parameters such as the angle and focal length of the monitoring device to ensure that the image of the target toad can be clearly captured, so as to resume the tracking and visual monitoring of the target toad.

[0054] In the visual monitoring method for the growth state of toads in the whole life cycle provided in this embodiment, when the monitoring of the target toad is interrupted, first, according to each first toad contour image of the target toad, the suspected position area of the target toad is predicted. In this way, it can provide a general position for the subsequent positioning of the target toad and narrow the matching range. Then, according to each first toad contour image and the regional feature image of the suspected position area, the feature matching degree between each candidate toad and the target toad in the suspected position area is determined; at the same time, according to the second toad contour images of each reference toad in the toad group to which the target toad belongs, the growth trajectory compliance degree between each candidate toad and the target toad in the suspected position area is determined. In this way, relying on the reference toads with a growth trajectory similar to that of the target toad and the feature matching situation between each candidate toad and the target toad in the suspected position area, the position of the target toad in the suspected position area can be accurately determined. To sum up, when the monitoring of the target toad is interrupted due to occlusion or other reasons during the toad monitoring process, the target toad can also be accurately tracked, thereby improving the accuracy of toad growth state monitoring.

[0055] As an optional embodiment, S102 may specifically include:

[0056] When the monitoring of the target toad is interrupted, according to each first toad contour image, the motion vectors of each pixel point of the target toad between two adjacent first toad contour images in time are determined by using the optical flow algorithm;

[0057] According to the motion vectors of each pixel point of the target toad, the suspected displacement of the target toad is predicted;

[0058] According to the position of the target toad before the monitoring interruption and the suspected displacement, the suspected position area of the target toad is determined.

[0059] In this embodiment, first, the server preprocesses two adjacent frames of first toad contour images in time, including denoising, grayscale conversion, etc., to ensure the accuracy of subsequent optical flow calculations.

[0060] Then, apply an optical flow algorithm (such as the Lucas-Kanade algorithm, Horn-Schunck algorithm, etc.) between these two frames of images to obtain the local motion vectors of each pixel point of the target toad between these two frames of images. Among them, the basic idea of the optical flow algorithm is to estimate the motion of an object by comparing the brightness changes of pixels between adjacent frames in an image sequence. For the Lucas-Kanade algorithm, it assumes that the motion of all pixel points within a small window is consistent, and then solves for the motion vector by minimizing the brightness error function; for the Horn-Schunck algorithm, it introduces a global smoothness constraint, assuming that the motion field changes smoothly, and obtains the motion vector by solving an energy minimization problem.

[0061] Then, since the target toad may be composed of multiple pixel points, its overall motion can be approximated by the average or weighted values of the motion vectors of these pixel points. Thus, according to the average or weighted values of the motion vectors of these pixel points, the global motion vector between these two frames of images is obtained. Among them, when performing weighted averaging, weights can be assigned according to the distance of the pixel points from the center of the target toad.

[0062] Then, after averaging the global motion vectors corresponding to two adjacent frames of the first toad contour images at each time and then converting them into the displacement amount of the target toad, the suspected displacement amount of the target toad can be obtained.

[0063] Finally, add the last position before the monitoring interruption to the calculated suspected displacement amount to obtain the predicted position of the target toad after the monitoring interruption. Since there may be errors in motion prediction, instead of predicting only one point as the position of the target toad, a region is predicted. This region can be a rectangle, ellipse, or other shaped region centered on the predicted position with a certain size and shape. Among them, the size and shape of the region can be determined according to factors such as the uncertainty of the displacement amount and the moving speed of the target toad.

[0064] Through this embodiment, in the case of monitoring interruption, the suspected position region of the target toad is predicted based on the optical flow algorithm. Thus, the region where the target toad is located can be roughly estimated, which can reduce the calculation amount for subsequent resuming tracking and improve the efficiency of resuming tracking of the target toad.

[0065] As an alternative embodiment, S103 may specifically include:

[0066] For each candidate toad in the suspected position region, perform the following steps respectively:

[0067] For each first toad contour image and region feature image, extract the local descriptors of the key feature points. The local descriptors are used to describe the feature information of the local region of the key feature points;

[0068] For each first toad contour image, respectively obtain the average value of the first Euclidean distances between the local descriptors of the key feature points of the first toad contour image and the local descriptors of the corresponding key feature points in the regional feature image;

[0069] Use the average values of the first Euclidean distances to determine the feature matching degree between the candidate toad and the target toad.

[0070] In this embodiment, the local descriptor is used to describe the feature information of the local area of the key feature point. Specifically, the local descriptor can be extracted by the Scale-Invariant Feature Transform (SIFT) feature extraction algorithm.

[0071] As an example, the feature matching degree between the candidate toad and the target toad can be specifically determined by the following formula 2:

[0072]

[0073] In formula 2, Z i is used to represent the feature matching degree between the i-th candidate toad and the target toad, g i,n is used to represent the number of frames between the i-th candidate toad in the regional feature image and the n-th first toad contour image before occlusion of the target toad, D i,n is used to represent the average value of the first Euclidean distances between the i-th candidate toad in the regional feature image and the n-th first toad contour image before occlusion of the target toad. N is used to represent the total number of first toad contour images, and exp is used to represent the exponential function operation.

[0074] Among them, is used to represent that the fewer the number of frames between the i-th candidate toad in the regional feature image and the n-th first toad contour image before occlusion of the target toad, the greater the reference significance of its average value of the first Euclidean distance, that is, the closer the first toad contour image to the regional feature image after occlusion in time, the more reference significance it has.

[0075] Through this embodiment, according to each first toad contour image and the regional feature image of the suspected position area, the feature matching degree between each candidate toad and the target toad in the suspected position area can be accurately determined. Thereby, it helps to accurately locate the target toad according to the feature matching degree subsequently, thereby improving the accuracy of toad growth state monitoring.

[0076] As an alternative embodiment, as Figure 2 shown, S104 can specifically include the following S201 to S203:

[0077] S201. For each reference toad, perform the following respectively: Based on each second toad contour image of the reference toad, determine the growth state change rate of the reference toad in each second toad contour image.

[0078] S202. For each reference toad, perform the following respectively: Based on the growth state change rate of each reference toad and the average value of the second Euclidean distances between the local descriptors of the key feature points in the second toad contour image of the reference toad and the local descriptors of the corresponding key feature points in the first toad contour image of the target toad, determine the growth state referenceability of the reference toad.

[0079] S203. Based on the growth trajectory conformity between the candidate toad and each reference toad and the growth state referenceability of each reference toad, determine the growth trajectory conformity between the candidate toad and the target toad.

[0080] In this embodiment, the growth state change rate is used to characterize the change rate of the contour of the reference toad in each second toad contour image.

[0081] The growth state referenceability is used to characterize the degree to which the growth state of the reference toad can be used as a reference for the target toad.

[0082] The second toad contour image is used to characterize the toad contour image corresponding to the reference toad.

[0083] As an alternative embodiment, the server first collects the second toad contour images of each reference toad at different time points. For each second toad contour image, use an image processing algorithm (such as edge detection, contour tracking, etc.) to extract the contour of the toad.

[0084] Then, for each second toad contour image, find the corresponding key feature points in it and the previous second toad contour image through a feature point matching algorithm (such as SIFT, SURF, etc.). According to the matched key feature points, calculate the displacement of each key feature point at adjacent times, and then calculate the change rate of the entire contour, that is, the growth state change rate of the reference toad in the second toad contour image. Specifically, this can be achieved by calculating the average value, standard deviation or other statistics of the key feature point displacements.

[0085] Then, for the second toad contour image of each reference toad and the first toad contour image of the target toad, calculate the local descriptors of the key feature points. By calculating the average value of the Euclidean distances between the local descriptors of the corresponding key feature points, measure the similarity between the two. Then, in combination with the growth state change rate of the reference toad and the average value of the second Euclidean distances, use methods such as weighted summation, multiplication or other combination methods to determine the growth state referenceability of the reference toad.

[0086] Finally, for each candidate toad, calculate the growth trajectory similarity between it and each reference toad. Specifically, this can be achieved by comparing the contour changes of the candidate toad and the reference toad at different time points, and using similarity metrics (such as contour matching, shape context, etc.) to evaluate the growth trajectory similarity. Then, according to the reference degree of the growth state of each reference toad, perform a weighted sum of the growth trajectory similarities between the candidate toad and each reference toad, so as to obtain the compliance degree of the growth trajectory between the candidate toad and the target toad. Among them, the weight can be determined according to the reference degree of the growth state of the reference toad, and the higher the reference degree, the greater the weight.

[0087] Through this embodiment, according to the second toad contour images of each reference toad in the toad group to which the target toad belongs, determine the compliance degree of the growth trajectory between each candidate toad and the target toad in the suspected position area. In this way, relying on the reference toads with similar growth trajectories to the target toad as references, the compliance degree of the growth trajectory between the candidate toad and the target toad can be accurately calculated. This helps to accurately screen the candidate toads according to the compliance degree of the growth trajectory subsequently, thereby improving the accuracy of toad growth state monitoring.

[0088] As an alternative embodiment, S201 may specifically include:

[0089] Obtain the straight-line distances of each key point pair of the reference toad in the target second toad contour image and the adjacent toad contour images of the target second toad contour image, where the key point pair is a pair of points formed by any two key feature points;

[0090] Use the straight-line distances of each key point pair in the target second toad contour image and the adjacent toad contour images of the target second toad contour image to determine the growth state change rate of the reference toad in the target second toad contour image.

[0091] In this embodiment, the growth state change rate of the reference toad in the target second toad contour image can be specifically determined by the following formula 3:

[0092]

[0093] In formula 3, S is used to represent the growth state change rate, T is used to represent the number of key point pairs. R is used to represent the number of images, that is, the number of the target second toad contour image and the adjacent toad contour images of the target second toad contour image. Among them, it is possible to preset the second toad contour images of the first five frames and the last five frames before and after the target second toad contour image as the adjacent toad contour images of the target second toad contour image. L t,r is used to represent the straight-line distance of the t-th key point pair in the r-th frame image, L t,r+1 is used to represent the straight-line distance of the t-th key point pair in the r + 1-th frame image.

[0094] Among them, It is used to characterize the difference between the distance ratio of key point pairs of toads in adjacent image frames and 1. The larger this value is, the greater the morphological change of the toad, that is, the greater the growth state change rate of the toad.

[0095] Through this embodiment, by using the straight-line distances of each key point pair of the toad in the target second toad contour image and the adjacent toad contour images of the target second toad contour image, the growth state change rate of the reference toad in the target second toad contour image can be accurately determined. In this way, it helps to determine the reference of the growth state of the reference toad according to the growth state change rate subsequently, thereby improving the accuracy of toad growth state monitoring.

[0096] As an alternative embodiment, S202 may specifically include:

[0097] Based on the growth state change rates of each reference toad, a growth state change rate sequence of the reference toad is constituted;

[0098] Based on the average value of the second Euclidean distances between the local descriptors of the key feature points of the second toad contour image of the reference toad and the local descriptors of the corresponding key feature points in the first toad contour image of the target toad, a descriptor distance sequence is constituted, and the descriptor distance sequence includes the average values of the second Euclidean distances.

[0099] Using the growth state change rate sequence and the descriptor distance sequence, the growth state reference of the reference toad is determined.

[0100] In this embodiment, the growth state change rate sequence includes the growth state change rates of the reference toad in each second toad contour image.

[0101] The descriptor distance sequence includes the average values of the second Euclidean distances between the local descriptors of the key feature points of each second toad contour image and the local descriptors of the corresponding key feature points in each first toad contour image of the target toad.

[0102] As an example, the growth state reference of the reference toad can be specifically determined by the following formula 4:

[0103] Q w = ρ(S w , exp(-D w )) Formula 4

[0104] In formula 4, Q w is used to characterize the growth state reference of the w-th reference toad, S w is used to characterize the growth state change rate sequence of the w-th reference toad, D wThe descriptor distance sequence used to characterize the w-th reference toad, and ρ is used to characterize the Pearson correlation coefficient.

[0105] Among them, the stronger the correlation between the growth state change rate sequence of the w-th reference toad and the descriptor distance sequence of the w-th reference toad, the greater the reference value of the growth state of the w-th reference toad.

[0106] Through this embodiment, using the growth state change rate sequence composed of each growth state change rate and the descriptor distance sequence composed of each average second Euclidean distance, the reference value of the growth state of the reference toad can be accurately evaluated. Thus, it helps to determine the growth trajectory compliance between the candidate toad and the target toad according to the reference value of the growth state subsequently, and can improve the accuracy of toad growth state monitoring.

[0107] As an alternative embodiment, S203 may specifically include:

[0108] Accumulate the growth state reference values of each reference toad to obtain a first accumulated value;

[0109] Accumulate the product of the growth state reference value of each reference toad and the corresponding growth trajectory similarity to obtain a second accumulated value;

[0110] Use the second accumulated value and the first accumulated value to determine the growth trajectory compliance between the candidate toad and the target toad.

[0111] In this embodiment, the growth trajectory compliance between the candidate toad and the target toad can be specifically determined by the following formula 5:

[0112]

[0113] In formula 5, Q w is used to characterize the growth state reference value of the w-th reference toad, X i,w is used to characterize the growth trajectory similarity between the i-th candidate toad and the w-th reference toad, W is used to characterize the number of reference toads, and F i is used to characterize the growth trajectory compliance between the i-th candidate toad and the target toad.

[0114] Among them, the greater the growth trajectory similarity between the candidate toad and each reference toad, the greater the growth trajectory compliance between the candidate toad and the target toad; the greater the growth state reference value of the reference toad, the more reference significance the growth trajectory similarity between the candidate toad and the reference toad has.

[0115] Through this embodiment, by using the growth trajectory similarity between the candidate toad and each reference toad, as well as the reference of the growth state of each reference toad, the growth trajectory compliance between the candidate toad and the target toad is accurately determined. This helps to accurately locate the target toad according to the growth trajectory compliance subsequently, thereby improving the accuracy of toad growth state monitoring.

[0116] As an alternative embodiment, as Figure 3 shown, before S104, the visual monitoring method for the growth state of the whole life cycle of toad farming may further include the following S301 to S302:

[0117] S301, determine the growth trajectory similarity between the first toad and the second toad according to the third toad contour image of the first toad and the fourth toad contour image of the second toad;

[0118] S302, perform density clustering based on each growth trajectory similarity to obtain at least one toad group.

[0119] In this embodiment, the first toad and the second toad are any two toads, the third toad contour image is the toad contour image corresponding to the first toad, and the fourth toad contour image is the toad contour image corresponding to the second toad.

[0120] As an example, the server compares the contour changes between the third toad contour image of the first toad and the fourth toad contour image of the second toad, and uses a similarity metric (such as contour matching, shape context, etc.) to evaluate the growth trajectory similarity between the first toad and the second toad.

[0121] Then, select a density clustering algorithm, such as DBSCAN or HDBSCAN. And set the parameters of the clustering algorithm, such as the minimum number of samples (MinPts) and radius (Eps), etc. Then, according to the growth trajectory similarity between each toad, apply the selected clustering algorithm for clustering to obtain at least one toad group.

[0122] Through this embodiment, according to the growth trajectory similarity between each toad, density clustering is performed on the toads to obtain at least one toad group. In this way, each toad with a similar growth trajectory is included in the finally obtained toad group, which helps to accurately track the target toad depending on the reference toad with a similar growth trajectory to the target toad in the case of interruption of target toad monitoring subsequently, thereby improving the accuracy of toad growth state monitoring.

[0123] As an alternative embodiment, S301 may specifically include:

[0124] Match the third toad contour image of the first toad with the fourth toad contour image of the second toad to obtain a group of matching images, where the group of matching images includes the third toad contour image and the fourth toad contour image at the same moment;

[0125] Obtain the average value of the third Euclidean distance between the local descriptors of the key feature points of the third toad contour image in the group of matching images and the local descriptors of the corresponding key feature points in the fourth toad contour image;

[0126] Use the average values of the third Euclidean distances to determine the growth trajectory similarity between the first toad and the second toad.

[0127] In this embodiment, the group of matching images includes the third toad contour image of the first toad and the fourth toad contour image of the second toad at the same acquisition moment.

[0128] As an example, the growth trajectory similarity between the first toad and the second toad can be specifically determined by the following formula 6:

[0129]

[0130] In formula 6, X a,b is used to represent the growth trajectory similarity between the a-th toad and the b-th toad, is used to represent the average value of the average values of the third Euclidean distances between the a-th toad and the b-th toad. OD a,b,k is used to represent the average value of the third Euclidean distance between the a-th toad and the b-th toad in the k-th group of matching images, OD a,b,k+1 is used to represent the average value of the third Euclidean distance between the a-th toad and the b-th toad in the (k + 1)-th group of matching images. K is used to represent the number of groups of matching images, and exp is used to represent the exponential function operation.

[0131] Among them, |OD a,b,k - OD a,b,k+1 | is used to represent the difference between the average values of the third Euclidean distances in adjacent consecutive frame toad contour images. The larger this value is, the less similar the growth trajectories are, that is, the smaller the growth trajectory similarity between the two toads.

[0132] Through this embodiment, using the average value of the third Euclidean distance between the local descriptors of the key feature points of the third toad contour image of the first toad and the local descriptors of the corresponding key feature points in the fourth toad contour image of the second toad can accurately measure the growth trajectory similarity between the first toad and the second toad. Thus, it helps to accurately group each toad with a similar growth trajectory according to the growth trajectory similarity in the subsequent process.

[0133] As an alternative embodiment, after S106, the visual monitoring method for the full life cycle growth state of toad farming may further include:

[0134] Determine the growth state result of the target toad based on each first toad contour image of the target toad;

[0135] Compare the growth state result of the target toad with the standard growth model to obtain a state comparison result;

[0136] Trigger an alarm mechanism when the state comparison result indicates an abnormal growth state.

[0137] In this embodiment, the growth state result is used to quantify the growth state of the target toad, and the standard growth model includes the normal growth state results of toads at each stage.

[0138] The server calculates features related to the growth state of the target toad based on each first toad contour image of the target toad, such as body length, body width, body weight, etc., so as to obtain the growth state result of the target toad. These features can be obtained through image measurement techniques, such as pixel counting, ratio calculation, etc.

[0139] Then, based on the growth law and historical data of toads, a standard growth model is established. This standard growth model can include a growth curve based on time, the relationship between body weight and body length, etc. Then, the growth state result of the target toad is matched with the standard growth model to evaluate whether the growth state of the target toad conforms to the standard growth model.

[0140] Finally, when it is determined that the growth state is abnormal, the alarm mechanism is immediately triggered. Specifically, it may include sending an email, text message or phone call to relevant personnel, or starting an automatic emergency response program, such as automatically adjusting the breeding environment, increasing the monitoring frequency, etc.

[0141] Through this embodiment, the growth state of the target toad is monitored based on each first toad contour image of the target toad. When the growth state is abnormal, the alarm mechanism is triggered in a timely manner, which helps to improve the effect of toad farming.

[0142] It should be clear that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, the detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps after understanding the spirit of the present invention.

[0143] It should also be noted that in the exemplary embodiments mentioned in the present invention, some methods or systems are described based on a series of steps or devices. However, the present invention is not limited to the order of the above steps. That is to say, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0144] As described above, the above is only the specific implementation manner of the present invention. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for visually monitoring the growth status of toads during their entire life cycle, characterized in that: The method comprises: Performing visual monitoring on a target toad and acquiring a first toad outline image of the target toad in real time; In the case where the monitoring of the target toad is interrupted, predicting the suspected location area of ​​the target toad according to each of the first toad outline images; Determining the feature matching degree between each candidate toad and the target toad in the suspected position area according to each of the first toad outline images and the regional feature image of the suspected position area; Determine the conformity of the growth trajectories of each candidate toad in the suspected position area with the target toad according to the second toad outline image of each reference toad in the toad group to which the target toad belongs, wherein the toad group includes toads with similar growth trajectories, and the reference toad is a toad in the toad group that is normally monitored except the target toad; Determining the overall matching degree between each of the candidate toads and the target toad according to the feature matching degree and the growth trajectory conformity degree; The candidate toad with the largest overall matching degree is determined as the target toad, and visual monitoring of the target toad is resumed.

2. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 1, wherein: In the case where the monitoring of the target toad is interrupted, predicting the suspected location area of ​​the target toad according to each of the first toad outline images includes: In the case where the monitoring of the target toad is interrupted, determining the motion vector of each pixel point of the target toad between the first toad outline images adjacent in time by an optical flow algorithm according to each of the first toad outline images; Predicting the suspected displacement of the target toad according to the motion vector of each pixel point of the target toad; The suspected position area of ​​the target toad is determined based on the position of the target toad before monitoring is interrupted and the suspected displacement.

3. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 1, wherein: The step of determining the feature matching degree between each candidate toad and the target toad in the suspected location area according to each of the first toad outline images and the regional feature image of the suspected location area comprises: For each candidate toad in the suspected location area, perform the following steps respectively: For each of the first toad outline images and the regional feature images, extract local descriptors of key feature points respectively, wherein the local descriptors are used to describe feature information of a local area of ​​the key feature points; For each of the first toad outline images, respectively obtain a first Euclidean distance average between a local descriptor of a key feature point of the first toad outline image and a local descriptor of a corresponding key feature point in the regional feature image; The feature matching degree between the candidate toad and the target toad is determined by using the average values ​​of the first Euclidean distances.

4. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 1, wherein: The step of determining the degree of conformity of the growth trajectories between each candidate toad and the target toad in the suspected position area according to the second toad outline image of each reference toad in the toad group to which the target toad belongs comprises: For each of the reference toads, respectively performing: determining the growth state change rate of the reference toad in each of the second toad outline images according to each of the second toad outline images of the reference toad; For each of the reference toads, respectively performing: determining the growth state reference of the reference toad according to the growth state change rate of each of the reference toads and the second Euclidean distance average between the local descriptors of the key feature points of the second toad outline image of the reference toad and the local descriptors of the corresponding key feature points in the first toad outline image of the target toad; The degree of conformity of the growth trajectory between the candidate toad and the target toad is determined based on the similarity of the growth trajectory between the candidate toad and each of the reference toads, as well as the reference of the growth status of each of the reference toads.

5. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 4, characterized in that: The step of determining the growth state change rate of the reference toad in each of the second toad outline images according to each of the second toad outline images of the reference toad comprises: Obtaining straight-line distances of each key point pair of the reference toad in the target second toad outline image and the adjacent toad outline image of the target second toad outline image, wherein the key point pair is a point pair formed by any two of the key feature points; The growth state change rate of the reference toad in the target second toad outline image is determined by using the straight-line distances of each key point pair in the target second toad outline image and the adjacent toad outline images of the target second toad outline image.

6. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 4, characterized in that: Determining the reference of the growth state of the reference toad according to the growth state change rate of each of the reference toads, and the second Euclidean distance average between the local descriptors of the key feature points of the second toad outline image of the reference toad and the local descriptors of the corresponding key feature points in the first toad outline image of the target toad, comprises: Based on the growth state change rates of the reference toads, a growth state change rate sequence of the reference toads is formed; Based on the second Euclidean distance averages between the local descriptors of the key feature points of the second toad outline image of the reference toad and the local descriptors of the corresponding key feature points in the first toad outline image of the target toad, a descriptor distance sequence is formed, wherein the descriptor distance sequence includes each of the second Euclidean distance averages; The growth state reference of the reference toad is determined by using the growth state change rate sequence and the descriptor distance sequence.

7. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 4, characterized in that: Determining the conformity of the growth trajectory between the candidate toad and the target toad based on the similarity of the growth trajectory between the candidate toad and each of the reference toads, and the reference of the growth status of each of the reference toads, comprises: Accumulating the growth status references of the reference toads to obtain a first accumulated value; Accumulating the product of the growth state reference of each reference toad and the corresponding growth trajectory similarity to obtain a second accumulated value; The second accumulated value and the first accumulated value are used to determine the degree of consistency of the growth trajectories between the candidate toad and the target toad.

8. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 1, characterized in that: Before determining the growth trajectory conformity between each candidate toad and the target toad in the suspected position area based on the second toad outline image of each reference toad in the toad group to which the target toad belongs, the method further includes: Determining the similarity of growth trajectories between the first toad and the second toad according to the third toad outline image of the first toad and the fourth toad outline image of the second toad; Density clustering is performed based on the similarity of each growth trajectory to obtain at least one toad group.

9. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 8, characterized in that: The determining the similarity of the growth trajectories between the first toad and the second toad according to the third toad outline image of the first toad and the fourth toad outline image of the second toad comprises: Matching the third toad outline image of the first toad with the fourth toad outline image of the second toad to obtain a matching image group, wherein the matching image group includes the third toad outline image and the fourth toad outline image at the same time; Obtaining an average third Euclidean distance between local descriptors of key feature points of the third toad outline image in the matching image group and local descriptors of corresponding key feature points in the fourth toad outline image; The similarity of the growth trajectories between the first toad and the second toad is determined by using the average values ​​of the third Euclidean distances.

10. The method for visually monitoring the growth status of toads during their entire life cycle according to claim 1, characterized in that: After determining the candidate toad with the largest overall matching degree as the target toad and resuming visual monitoring of the target toad, the method further comprises: Determining the growth status result of the target toad according to each of the first toad outline images of the target toad; Comparing the growth state result of the target toad with the standard growth model to obtain a state comparison result; In the case where the status comparison result indicates that the growth status is abnormal, an alarm mechanism is triggered.

Images