A neural network automatic design method based on wild animal monitoring image classification

Abstract

The present application relates to a kind of wild animal monitoring image classification neural network automatic design method, comprising, S1, the framing of animal boundary, generate proxy dataset, input target pixel height, width, animal area coordinates;S2, training and verifying sampling network, the "strong delay monotonicity" between the verification search workstation and edge device;S3, based on proxy dataset, carry out neural network structure search based on low round number training, with workstation as the proxy device of edge device, measure network inference time;S4, based on approximate accuracy and approximate inference time, calculate segmented loss function, and optimize algorithm;S5, according to application demand, select network.The network data finally obtained by the present application through continuous calculation, optimization algorithm has the highest average accuracy and the lowest video memory occupancy rate compared with the rest of network data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wild animal monitoring, and particularly relates to a neural network automatic design method for wild animal monitoring image classification. BACKGROUND

[0002] Wild animals play an important role in maintaining ecological balance, and therefore, effective monitoring and protection of wild animals have important practical significance.

[0003] The "neural network structure search method" is a method for automatically designing a neural network structure based on task requirements. The search process generally includes sampling networks, sorting sampled networks, etc. The basic structural framework of the sampled network is derived from a set of networks with the same meta-structure, which is called a "search space". Generally, the search space contains a large number of networks, which are called "candidate networks", and the evaluation of the performance of the candidate networks is often very expensive.

[0004] The resolution of the task data set image is generally high, and the application of the search method on the data set is often costly. For large task data sets with skewed distribution, the proxy data set constructed based on the task data set is more beneficial to the search method than the public benchmark data set. Field monitoring equipment may generate millions of "monitoring images", and the number of images of each species differs greatly. Therefore, constructing a proxy data set based on monitoring images may be beneficial to the search method.

[0005] When the computing resources are limited, even if the search method has applied the proxy data set, the search cost is still high. Therefore, the performance of the network is often measured with low configuration settings, and experiments have shown that the image classification accuracy generated by the network based on different rounds of training has relatively high correlation. If the search method also applies low round training and proxy data set, the case based on "more samples and less round training" is more beneficial to the search method than the case based on "less samples and more round training".

[0006] For edge devices running convolutional neural networks, network inference time is critical. However, measuring inference time based on edge devices is costly for search algorithms. To reduce the cost of inference time measurement, various alternative methods have emerged. One method is to construct a delay lookup table based on the average time consumption of different inputs processed by the same operation before the search begins. During the search, the operation processing time of the sampled network is queried one by one through the delay lookup table, and then the obtained processing time is added to obtain the network inference time. Regardless of how the predictor is implemented, the predicted inference time not only has errors, but also is limited to a specific device and input. A more flexible alternative method for measuring inference time is to complete the measurement on the corresponding proxy device of the edge device. The inference time measured on the proxy device must have a strong correlation with the inference time measured on the edge device. The network involved in the calculation of the Spearman rank correlation coefficient is generally randomly sampled from the search space, and the randomly sampled network has a large number of similar inference time values, which reduces the reliability of the calculation of the Spearman rank correlation coefficient. Therefore, how to sample the network so that its inference time is widely distributed is still pending. SUMMARY

[0007] To this end, the present application provides a neural network automatic design method for wild animal monitoring image classification. To overcome the problem of low accuracy of data collection and unstable reliability of proxy device in the prior art.

[0008] To achieve the above-mentioned purpose, the present application provides a neural network automatic design method for wild animal monitoring image classification, comprising,

[0009] S1, framing of animal boundary, generating proxy data set, inputting target pixel height, width, and animal region coordinates;

[0010] S2, training and verifying the sampling network, verifying the "strong delay monotonicity" between the search workstation and the edge device;

[0011] S3, based on the proxy data set, carrying out neural network structure search based on low round training, taking the workstation as the proxy device of the edge device, and measuring the network inference time;

[0012] S4, calculating the segmented loss function based on the approximate accuracy and the approximate inference time, and optimizing the algorithm;

[0013] S5, selecting the network according to the application requirement.

[0014] Neural architecture search based on reinforcement learning theory is developed to train an agent to complete a task in a way that rewards the agent for beneficial task behavior. In neural architecture search based on reinforcement learning theory, the "agent" is typically a long short-term memory (LSTM) network, and a convolutional neural network is constructed based on the sampled words, which is referred to as a "sampled network".

[0015] Further, the optimization algorithm is implemented by optimizing the agent through rewards and improving the behavior of the agent. The loss function involved in the optimization algorithm is calculated based on the performance of the LSTM sampled network. The performance of the sampled network mainly includes the approximation accuracy and the approximation inference time, that is, based on the training set, the network is trained with a low number of rounds, the trained network processes the validation set to obtain the accuracy and the total processing time, if the accuracy does not exceed the predefined threshold, the current validation accuracy is the approximation accuracy, and the current inference time is the approximation inference time, otherwise, the training and testing are repeated based on different random seeds to generate multiple validation accuracies and inference times, and the multiple validation accuracies are fitted through a metalog distribution, the median of the metalog distribution, and the mean of the multiple inference times. The device used for processing the validation set by the network is a proxy device of an edge device.

[0016] Further, based on the approximation accuracy and the approximation inference time, the loss value is calculated through the loss function, and the obtained loss value is combined with the optimization algorithm, that is, the LSTM parameters are optimized. The above process is repeated until the number of networks sampled by the LSTM reaches the preset number limit, and the above process is referred to as "search". In this process, all the networks sampled by the LSTM and their performance, network structure and other related information are saved in a MySQL database in the order of sampling time, and all or part of the network related information sampled by the same LSTM saved in the database is referred to as "search history".

[0017] Further, according to the search history, a network is selected, which is the search output and also referred to as "final network".

[0018] Compared with the prior art, the beneficial effects of the present application are that the network classification monitoring image result obtained through continuous calculation and optimization algorithm is more accurate.

[0019] The quantile calculation proposed in the present application improves the precision of fitting a univariate Gaussian distribution of the metalog distribution under the condition of low sampling number.

[0020] The present application improves the network sampling method required in the identification process of the proxy device, improves the difference degree of the inference time of the sampled network, and strengthens the calculation reliability of the identification process. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 Flow chart of neural network automatic design method for wild animal monitoring image classification in embodiments of the present application;

[0022] Figure 2 Random search historical data graph based on NACTI-b proxy dataset;

[0023] Figure 3 Network sampling graph based on random search;

[0024] Figure 4 Binary image based on random search network sampling;

[0025] Figure 5 Error mean curve graph when n=2;

[0026] Figure 6 Error mean curve graph when n=3;

[0027] Figure 7 Error mean curve graph when n=4;

[0028] Figure 8 Error mean curve graph when n=5;

[0029] Figure 9 Search historical data graph of the method of the present application;

[0030] Figure 10 Convolution and pooling unit structure diagram. DETAILED DESCRIPTION

[0031] In order to make the objects and advantages of the present application clearer, the present application will be further described below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0032] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0033] It should be noted that in the description of the present application, the terms "upper", "lower", "left", "right", "inner", "outer" and the like indicating the direction or positional relationship of the terms are based on the direction or positional relationship shown in the drawings, which is only for the convenience of description, and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0034] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0035] Please see Figure 1 The diagram shown is a flowchart of an automatic neural network design method for wildlife monitoring image classification in an embodiment of the present invention.

[0036] This invention provides an automatic design method for neural networks for classifying wildlife monitoring images, including:

[0037] S1, defining the boundaries of the animals, generating a proxy dataset, inputting the target pixel height and width, and the coordinates of the animal region;

[0038] S2, train and validate the sampling network to search for strong latency monotonicity between workstations and edge devices;

[0039] S3, based on the proxy dataset, conducts neural network architecture search based on low-round training, and uses workstations as proxy devices at the edge to measure network inference time;

[0040] S4. Based on the approximate accuracy and approximate inference time, calculate the piecewise loss function and optimize the algorithm;

[0041] S5 selects networks based on application requirements.

[0042] In step S1, the animal areas in the field monitoring images were defined.

[0043] For a field monitoring image K, if there are animal regions in K, these animal regions are numbered and denoted as b1, b2...b n Then, based on each of the animal regions, corresponding images P1, P2...P are generated. n Where, for any image P i Its width is w t The height is h t , i = 1, 2, ... n;

[0044] For any animal region b i If its width and height are the same as P i The required width and height are different, which applies to animal area b. i The width and height are scaled and adjusted to generate a value with a width of w. t and height is h timage P i . The scaling method is: according to w t / h t ratio and b i width and height, generate empty image P i ' conforming to the ratio, and b i long side is equal to the corresponding side length of P i ', copy b i to the central area of P i ', and then scale to obtain P t with width w t and height h i by interpolation algorithm.

[0045] In step S3, the parameter quantity s of the candidate network is sampled from the search space by LSTM, s1≤s≤s2, wherein s2 is the upper limit of the preset parameter quantity, and s1 is the lower limit of the preset parameter quantity. When s

[0046]

[0047] s num is the preset number of sampling failures, and s is the upper limit of the number of sampling failures. If s , the resampling is terminated.

[0048] In step S2, in order to reduce the interference of low round training on the search method, the existence of strong delay monotonicity between the search workstation and the edge device is verified by using the metalog distribution and the accuracy of the sampling network, and the network inference time is measured by taking the workstation as the proxy device of the edge device. If the verification accuracy is higher than the preset value A, the network is repeatedly trained and verified m times according to different random seeds to obtain accuracies a1, a2...a m . The distribution of the above m accuracies is fitted by using the metalog distribution, wherein the necessary quantile of the metalog distribution is calculated based on the accuracy histogram, and the histogram group number h * is determined by the histogram group number optimization algorithm.

[0049] The histogram group distance b is calculated according to the histogram group number h * , that is:

[0050]

[0051] wherein ε=10 -6 , a max represents the maximum value in a1, a2...a m , and a min represents the minimum value in a1, a2...a mthe minimum value in the set of accuracies. In a histogram with bin width b, the accuracies a1, a2...a m each accuracy in the set of accuracies lies in a bin in the histogram, the jth accuracy in the kth bin is denoted as aj,k k，j which satisfies the following equation:

[0052] a min +(k-1)b≤a k，j -1≤a k，j ≤a k，j +1<a min +kb

[0053] a k，j the quantile y of the set of accuracies k，j is calculated according to the following equation:

[0054]

[0055] b k is the number of accuracies in the kth bin, and the metalog distribution fits the accuracy distribution according to the quantiles calculated above, and the median of the fitted distribution is the approximate accuracy.

[0056] n' candidate networks are randomly sampled from the search space, and their image classification accuracies and inference times are evaluated on the aforementioned proxy dataset by a high-performance device. In a two-dimensional coordinate system with the accuracy as the horizontal axis and the inference time as the vertical axis, each of the n' candidate networks is represented by a point (a, t), where a represents the accuracy of the network and t represents the inference time of the network.

[0057] On the aforementioned accuracy and inference time plane, for a convex hull containing n' points, the Euclidean distance between the vertices of the convex hull is large. If the number of vertices of the convex hull is greater than m', then all the vertices of the convex hull are sampled and the sampling ends, otherwise after sampling the vertices of the convex hull, the inside of the convex hull is moved in the accuracy and inference time plane, the points moved through are deleted, the convex hull of the remaining points is calculated and the vertices thereof are sampled, the moving and sampling are repeated until the number of sampled points is not less than m', and the moving and sampling process is realized.

[0058] According to the distribution of the n' points on the accuracy and inference time plane, a binary image B0 of size n' x n' is constructed, and any point (a, t) in the n' points corresponds to a pixel p(x, y) = 1 in B0 by the following equation, and the rest of the pixels are 0.

[0059]

[0060] where a min is the minimum value of the accuracy, a max is the maximum value of the accuracy, and tmin t represents the minimum inference time. max This represents the maximum inference time. Based on the positions of the n′ non-zero pixels in B0, calculate the convex hull H0 that surrounds these pixels. The candidate networks corresponding to the vertices of this convex hull are stored in the set... middle.

[0061] like The base number is smaller than m′, that is... Image erosion is then performed based on B0, and the resulting image is denoted as B1. Based on the non-zero pixels in B1, the convex hull H1 enclosing these pixels is calculated, and the candidate networks corresponding to the vertices of this convex hull are stored in a set. middle.

[0062] During the convex hull computation, the network corresponding to the convex hull vertices is saved, and image erosion is performed as needed, repeating this process continuously until... The number of true or non-zero pixels is 0.

[0063] The detailed steps of image erosion are described below.

[0064] If the I-th image erosion is completed Established and B I If there are still non-zero pixels in the matrix, then based on B... I The (I+1)th image erosion is performed, using an "intra-image rectangle" as the structuring element, where the width and height of the intra-image rectangle are s. x With s y From vector s = [s x s y ] T Definition, that is:

[0065]

[0066] in express Cardinality; abs(·) indicates calculating the absolute value; v is the connection between B and B. I Image coordinates The vectors of (x′, y′)

[0067]

[0068] B I Mean image coordinates of non-zero pixels, convex hull H I In the vertex The vertex with the smallest Euclidean distance is denoted as p(x′, y′).

[0069]

[0070] ||·|| denotes the L2 norm of a vector. Based on the structuring element and on B... I Perform image erosion,

[0071]

[0072] Represents a binary image of size n′×n′; ~ represents The image coordinates of non-zero pixels are only the same as the image coordinates of pixels in the set on the right side of the above equation; Represents a binary image of size n′×n′, whose non-zero elements are limited to H. I The surrounding pixels. For The pixel p(x, y) is equivalent to verification. The middle part is centered at p(x,y) and has a size of s. x ×s y Does the rectangular area contain zero pixels? If it does not contain zero pixels, then... In the image, the pixel with coordinates (x, y) is set to 1, otherwise it is set to zero. B I Image B after erosion I+1 The definition is as follows:

[0073]

[0074] ∩ represents an element-wise logical AND operation, based on B. I The (I+1)th image erosion is completed.

[0075] Calculate the bounding box B I+1 Convex hull H of non-zero pixels I+1 Save the candidate network corresponding to its vertex to like Then repeat the image erosion, i.e., based on B. I+1 Perform the (I+2)th image erosion step; otherwise, terminate the image erosion and return.

[0076] Design a piecewise loss function based on approximate accuracy and approximate inference time. Update the LSTM by optimizing the algorithm.

[0077] loss function The definition is as follows:

[0078]

[0079] a represents the approximate accuracy of the network sampled by the LSTM, in %; t represents the approximate inference time of the network sampled by the LSTM, in seconds; * This represents the ideal accuracy rate, expressed in %; t *denotes an ideal inference time in seconds; t1 denotes a threshold value 1 defined by the actual requirements, and t2 denotes a threshold value 2 defined by the actual requirements;

[0080] η is a constant and is defined as follows

[0081]

[0082] In the above formula, sigma is the same variable, which is:

[0083]

[0084] After the search is completed, the network is selected from the candidate network sampled by the LSTM as the output.

[0085] Referring to Figures 2-10 as shown, Figure 2 is a random search history data graph based on the NACTI-b proxy data set; Figure 3 is a network sampling graph based on random search; Figure 4 is a binary image based on random search network sampling; Figure 5 is an error mean curve graph when n = 2; Figure 6 is an error mean curve graph when n = 3; Figure 7 is an error mean curve graph when n = 4; Figure 8 is an error mean curve graph when n = 5; Figure 9 is a search history data graph of the method of the application; Figure 10 is a convolution and pooling unit structure diagram.

[0086] One specific embodiment of the neural network automatic design method for wild animal monitoring image classification:

[0087] According to the 26689 animal regions labeled in the NACTI-b training set image subset of the North American wild animal monitoring image data set, the proxy data set image is generated by the algorithm in S1, the resolution is the same as the public benchmark data set CIFAR-10, the number of images is the same as the number of animal regions, and 20% of the images are randomly selected to form a validation set. In the proxy data set, the number of training set images of each category is shown in Table 1.

[0088] Table 1 Number of samples in NACTI-b proxy data set

[0089]

[0090]

[0091] To verify that the workstation and the edge device have "strong delay monotonicity", a random search is carried out on the workstation based on the verified agent data set, the network structure sampled by the random search, and the training accuracy, verification accuracy, parameter quantity, inference time (verification set processing time consumption) and other information are saved in the MySQL database. According to the data recorded in the database, the random search history scatter plot is drawn, as shown in Figure 2 , the random search takes about 40 hours and 30 minutes, and each point in the figure corresponds to a search-sampled candidate network, and the horizontal and vertical coordinates of the point correspond to the verification accuracy of the corresponding network and the total time consumption of processing the verification set.

[0092] Through S2, according to the network sampling number m', the network with large inference time difference is automatically sampled. The sampling number of the present application is set to m' = 100, and the sampling is completed by 5 times of S2, and the sampled network corresponds to Figure 3 The vertices of the convex hull H0, H1, H2, H3 and H4. Based on the binary image composed of H0, H1, H2, H3 and H4 , as shown in Figure 4 .

[0093] Copy the verification set of the sampled network in S2 and the NACTI-b agent data set to the edge device, and record the time consumption of each network processing the verification set. The time consumption of each network processed by the workstation based on S2 and the time consumption of each network processed by the edge device are arranged in the same order to form two vectors, and the Spearman rank correlation coefficient of the two vectors is 0.9657, which meets the "strong delay monotonicity" requirement. Therefore, the workstation and the edge device have "strong delay monotonicity".

[0094] The bounded metalog distribution defined by the method of the present application mainly includes the parameter number n, the lower bound b0 of the fitted data and the upper bound b1 of the fitted data. The present application sets b0 = x min -0.1 and b1 = x max +0.1, wherein x min is the minimum value of the fitted data, and x max is the maximum value of the fitted data. The larger the parameter number n, the more complex the metalog fitting degree distribution, but the required data quantity also increases accordingly. For the method of the present application, the data required by the bounded metalog distribution is based on different random seeds, and the same network is trained and verified multiple times, and multiple accuracies are obtained. Therefore, increasing the number of training and testing networks m has a high calculation cost, and the accuracy obtained by testing the network is subject to a univariate normal distribution, so the value of n should be considered according to the different values of m. The value range of the natural number m is set as m ∈ [20, 101].

[0095] If the number of bounded meta log distribution parameters is n, for any value of m in [20, 101], two values are randomly sampled from two independent uniform distributions with boundaries 0 and 10 as the mean and variance of a univariate normal distribution, which is called a “randomized normal distribution”. By sampling m values from the randomized normal distribution, b0 and b1 are calculated based on the m sampling values, and m quantiles are calculated by S3, and the hyperparameters of the bounded meta log distribution coefficients b0, b1 and n are calculated based on the m quantiles, and the distribution expression is obtained, which is called “metalog fitting distribution”, and the calculation process of the meta log fitting distribution is called “metalog distribution fitting m sampling values”. According to the meta log distribution expression calculated by the above calculation, the difference between the median determined by the distribution expression and the mean of the randomized normal distribution is the “fitting error”. For each pair of values of n and m, repeat the calculation of the fitting error 20 times, and the average of these fitting errors is called “fitting error average”.

[0096] Figure 5 When n = 2 and m ∈ [20, 101], the curve of the fitting error average calculated based on the quantile calculation method described in S3 is arranged according to the value of m, which is called “error average curve”. Figure 6 The error average curve when n = 3 and m ∈ [20, 101] is shown; Figure 7 The error average curve when n = 4 and m ∈ [20, 101] is shown; Figure 8 The error average curve when n = 5 and m ∈ [20, 101] is shown. Figure 9 As shown, when the sampling number m ≤ 45 and n ≤ 4, the fitting error average is significantly lower than the fitting error average calculated based on the meta log distribution Python implementation. Therefore, for a given convolutional neural network, the accuracy of the network obtained by m = 20 times of training and verification (before each training, the network is randomly initialized) is used as the sampling value of the accuracy distribution of the network, and b0 and b1 are calculated based on these sampling values. The meta log fitting distribution with hyperparameters b0, b1 and n = 4 fits m sampling values, and the median determined by the meta log fitting distribution is called “approximate accuracy”.

[0097] In S2 of the present application, the upper and lower limits of the parameter quantity s1 and s2 are set to s1 = 1.5M and s2 = 3M (M represents million). According to the random search result, the hyperparameters t * i.e. the ideal reasoning time, is set to Figure 10 In the search history shown, the average reasoning time of the network between s1 and s2 is 3.3 seconds, i.e. t * = 3.3 seconds. In S4, other hyperparameters are set to t1 = 1, t2 = 1.50, a *= 100 (ideal accuracy 100%) and σ = 0.67423.

[0098] The accuracy threshold A is set to Figure 2 The integer part of the minimum value (in %) of the verification accuracy in the first 20% of the search history is A = 49, and the upper limit of the total number of sampling networks is set to 1500.

[0099] The sampling history is shown in Figure 3 The figure, where the text labels, the meaning of each graphical element is the same as Figure 4 As shown in Figure 5 The method of the present application takes about 67.2 hours, and the final network corresponds to the star in the figure, which contains 3 pooling units, and the pooling units are arranged in order. There are 3 convolution units between each two pooling units, and in the pooling unit order, there are 3 convolution units before the first pooling unit, and 3 convolution units after the last pooling unit. All convolution units have the same structure but different parameters. All the pooling units have the same structure but different parameters. The structures of the convolution and pooling units are shown in Figure 6 Figure 7 Figure 8 Figures 5 to 8 Figure 1 Figure 1 Figure 9 Figure 2 Figure 9 Figure 10 .

[0100] The final network searched by the method of the present application is trained on the training set of the NACTI-b data set with a resolution of 224x168, and tested on the test set of the data set, where the workstation used for training and testing is the same as the workstation used for searching. Limited by the limited computing resources of the workstation, the training batch size of the final network is set to 8, and the training round number is set to 20.

[0101] In order to compare the method of the present application with other search methods, the final network of other search methods is trained on the training set of the NACTI-b data set with a resolution of 224x168, that is, the following six networks:

[0102] 1) NAO_V1

[0103] 2) SETN

[0104] 3) NASNet

[0105] 4) ENAS

[0106] 5) GDAS

[0107] 6) PNASNet

[0108] In addition to batch size and the number of rounds, other training settings of these networks are the same, wherein the batch size is uniformly set to 3 according to the condition of the network consuming the video memory. Because the training based on a lower batch size consumes more time than the training based on a higher batch size, the training rounds of the final network of other search methods are uniformly set to 15 to make the training time of the final network of other search methods close to the training time of the final network of the application. On the workstation, the test set images of the NACTI-b dataset with the resolution adjusted to 224x168 are calculated by the trained network to obtain labels, and according to the labels obtained by calculation and the labels of the test set images, the top-1 accuracy (this accuracy is called "test accuracy") is calculated. The test accuracy obtained (unit: %) and other test indicators (units are marked in the table) are shown in Table 2.

[0109] Table 2 Test accuracy comparison

[0110]

[0111]

[0112] Table 2 shows the inference time, parameter amount, video memory occupation, test accuracy of each species, and the average value of the test accuracy, i.e. "average accuracy" (unit: %) in the table, of each network described above, wherein the inference time is the average time consumed by the network to process the test set images with a batch size of 1; the parameter amount is obtained from the network parameters; the video memory occupation is the video memory occupation record obtained by calling the nvidia-smi program to view the GPU video memory occupation when the network processes the test set and the batch size is set to 1; the bold characters in each row in the table correspond to the optimal value of the index in the row. As shown in Table 2, the inference time of the method of the application is moderate (67 milliseconds), the parameter amount is the lowest (2.38 million) among the comparison networks, the video memory occupation is the lowest (3212 MB, MB is megabyte), and the average accuracy is the highest (97.26%).

[0113] So far, the technical solutions of the application have been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the application, and the technical solutions after these changes or replacements will fall within the protection scope of the application.

[0114] The above description is only for the preferred embodiments of the application and is not used to limit the application; for those skilled in the art, the application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. An automatic design method for neural networks based on wildlife monitoring image classification, characterized in that, Including, S1, framing the animal boundary, generating a proxy dataset, inputting the target pixel height, width, and the coordinates of the animal area; S2, training and validating the sampling network to check the strong delay monotonicity between the search workstation and the edge device; S3, based on the proxy dataset, conducting neural network architecture search with low-round training, and measuring the network inference time using the proxy device of the workstation as the edge device; S4, calculating the piecewise loss function based on the approximate accuracy and approximate inference time, and optimizing the algorithm; S5, selecting a network according to the application requirements; In step S2, to verify the strong delay monotonicity between the search workstation and the edge device, n' candidate networks are randomly sampled from the search space. Their image classification accuracy and inference time are obtained by evaluating the generated proxy dataset using a high-performance device. In the two-dimensional coordinate system with accuracy as the horizontal axis and inference time as the vertical axis, each of the n' candidate networks is represented by a point (a, t), where a represents the accuracy of the network and t represents the inference time of the network.

2. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 1, characterized in that, In step S1, the animal regions in the field monitoring images are defined; for field monitoring image K, if there are animal regions in K, the animal regions are numbered and denoted as b1, b2...b n Based on each of the animal regions, generate corresponding images P1, P2...P n Where, for any image P i Its width is w t The height is h t , i = 1, 2, ..., n.

3. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 2, characterized in that, For any animal region b i If its width and height are the same as P i The requirements for width and height differ, thus affecting animal area b. i The width and height are scaled and adjusted to generate a value with a width of w. t and h t For high-resolution image P i The scaling method is: based on w t / h t Proportion and b i Given width and height, generate an empty image P that conforms to these proportions. i ', and b i Long side and P i 'If the corresponding side lengths are equal, then b' i Copy to P i The centered area is then scaled using an interpolation algorithm to obtain a width of w. t and height is h t P i .

4. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 3, characterized in that, On the accuracy and inference time plane, for the convex hull containing n' points; if the number of vertices of the convex hull is more than m', then sample all the vertices of the convex hull and end the sampling. Otherwise, after sampling the vertices of the convex hull, move inward on the accuracy and inference time plane, delete the points passed by the movement, calculate the convex hull of the remaining points and sample its vertices, repeat the movement and sampling until the number of sampled points is not less than m', and the movement and sampling process are realized; According to the distribution of the n' points on the accuracy and inference time plane, construct a binary image B0 of size n'×n'. For any point (a, t) among the n' points, the pixel p(x, y) in B0 corresponding to it is 1 through the following formula, and the rest of the pixels are 0; Where a min a is the minimum accuracy. max t represents the maximum accuracy. min t represents the minimum inference time. max This represents the maximum inference time. Based on the positions of the n' non-zero pixels in B0, the convex hull H0 surrounding these pixels is calculated, and the candidate networks corresponding to the vertices of this convex hull are stored in a set. middle; like The base is smaller than m', that is Image erosion is then performed based on B0, and the resulting image is denoted as B1. Based on the non-zero pixels in B1, the convex hull H1 enclosing these pixels is calculated, and the candidate networks corresponding to the vertices of this convex hull are stored in a set. middle.

5. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 4, characterized in that, During the convex hull computation, the network corresponding to the convex hull vertices is saved, and image erosion is performed as needed, repeating this process continuously until... The number of valid or non-zero pixels is 0. The detailed steps of image erosion are as described below: If the I-th image erosion is completed Established and B I If there are still non-zero pixels in the matrix, then based on B... I The (I+1)th image erosion is performed, using an "intra-image rectangle" as the structuring element, where the width and height of the intra-image rectangle are s. x With s y From vector s = [s x ,s y ] T Definition, that is: in express Cardinality; abs(·) indicates calculating the absolute value; v is the connection between B and B. I Image coordinates The vectors of (x', y') B I The mean image coordinates of non-zero pixels, for Let's explain the definitions of the four data points x', y', and y' separately. The mean of the horizontal axis. x' is the mean of the vertical coordinates, x' is the value of the horizontal coordinate, and y' is the value of the vertical coordinate. Convex hull H I In the vertex The vertex with the smallest Euclidean distance is denoted as p(x',y'). ||·|| represents the L2 norm of a vector, based on the above structuring elements and on B. I Perform image erosion, Represents a binary image of size n'×n'; ~ represents The image coordinates of non-zero pixels are only the same as the image coordinates of pixels in the set on the right side of the above equation; Represents a binary image of size n'×n', whose non-zero elements are limited to H. I Enclosing pixels; for The pixel p(x,y) is equivalent to verification. The middle part is centered at p(x,y) and has a size of s. x ×s y Does the rectangular area contain zero pixels? If it does not contain zero pixels, then... In the image, pixels with coordinates (x, y) are set to 1, otherwise to zero; B I Image B after erosion I+1 The definition is as follows: ∩ represents an element-wise logical AND operation, based on B. I The (I+1)th image erosion is completed.

6. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 5, characterized in that, Calculate the bounding box B I+1 Convex hull H of non-zero pixels I+1 Save the candidate network corresponding to its vertex to like Then repeat the image erosion, i.e., based on B. I+1 Perform the (I+2)th image erosion step; otherwise, terminate the image erosion and return.

7. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 6, characterized in that, In step S3, the parameter quantity s of the candidate network sampled by LSTM from the space containing all sampled network structures, that is, the search space, satisfies s1 ≤ s ≤ s2, where s2 is the upper limit of the preset parameter quantity and s1 is the lower limit of the preset parameter quantity; when s < s1 or s2 < s, it is determined that the current sampling fails, and the network is resampled, and s1 and s2 are updated according to the following formula Where s num The preset number of sampling failures. This is the maximum number of sampling failures. like Then resampling will be terminated.

8. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 7, characterized in that, Using a workstation as a proxy device at the edge, the network inference time is measured. If the verification accuracy is higher than a preset value A (i.e., the smallest integer part of the top 20% verification accuracy in the random search history), then the network is trained and verified m1 times using different random seeds to obtain accuracies a1, a2…a1. m The distributions of the above m accuracies are fitted using the metalog distribution, where the necessary quantiles for the metalog distribution are calculated based on the accuracy histograms, and the number of histogram groups is h. * Determined by the histogram group number optimization algorithm; Based on the number of histogram groups h * Calculate the histogram group interval b, i.e.: Where ε = 10 -6 a max Represents a1, a2…a m The maximum value in, a min Represents a1, a2…a m The minimum value in the histogram with class interval b; the accuracy a1, a2…a m Each accuracy rate is located in a group within the histogram, and the j-th accuracy rate a in the k-th group... k,j The accuracy satisfies the following formula: a min +(k-1)b≤a k,j-1 ≤a k,j ≤a k,j+1 <a min +kb a k,j quantile y k,j Calculate according to the following formula: b k It is the number of accuracy rates in the k-th group. and The accuracy distribution of the metalog distribution is fitted based on the quantiles calculated by the above formula, and the median of the fitted distribution is the accuracy.

9. The method for automatically designing neural networks for wildlife monitoring image classification according to claim 8, characterized in that, A piecewise loss function was designed based on approximate accuracy and approximate inference time. Update the LSTM by optimizing the algorithm; loss function The definition is as follows: a represents the approximate accuracy of the network sampled by the LSTM, in %; t represents the approximate inference time of the network sampled by the LSTM, in seconds; * This represents the ideal accuracy rate, expressed in %; t * t1 represents the ideal inference time in seconds; t2 represents threshold 1 defined by actual needs; t2 represents threshold 2 defined by actual needs. η is a constant, defined as follows In the above formula, σ is the same variable, and this variable is: After the search ends, select a network as the output from the candidate networks sampled by LSTM.

Images