Image segmentation model training method and device, electronic equipment and storage medium
Patent Information
- Application Number
- CN202510845364.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-23
- Publication Date
- 2025-10-17
Smart Images
Figure CN120807915A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, and in particular to a training method and device of an image segmentation model, an electronic device and a storage medium. BACKGROUND
[0002] With the advancement of agricultural intelligence, automatic driving agricultural machines are gradually popularized in the application of agricultural production links such as plowing, seeding, fertilizing and harvesting. The key lies in real-time perception of farmland environment and intelligent decision-making. The image segmentation model based on deep learning can accurately identify crops, weeds and obstacles, and generate pixel-level segmentation results, providing accurate environmental perception support for automatic navigation. However, in the embedded system of agricultural machinery, the use of these models still faces many challenges. First, agricultural machinery mostly uses low-power embedded processors, which have limited computing power and are difficult to support real-time inference of high-complexity deep learning models, resulting in delayed segmentation results and affecting system response speed. Second, the farmland environment is highly unstructured, with complex factors such as light changes, dynamic occlusions, uneven vegetation density and soil reflection, which reduce the robustness of the model and affect the segmentation accuracy. Finally, the hyperparameters of the image segmentation model (such as learning rate, feature extraction layer number and mask segmentation scale) are crucial to the segmentation performance, while traditional hyperparameter search methods (such as grid search and random search) have large computational overhead and slow convergence speed, making it difficult to meet the simultaneous optimization requirements of real-time performance, computational efficiency and accuracy of the agricultural automatic driving system.
[0003] Current hyperparameter optimization algorithms for image segmentation models, such as Simulated Annealing (SA) and Artificial Bee Colony Algorithm (ABC), have certain limitations in handling hyperparameter optimization. These algorithms are prone to slow convergence speed and easy to fall into local optimum in complex high-dimensional search space, resulting in low optimization efficiency and difficulty in meeting the strict requirements of real-time performance and computational resources of the agricultural automatic driving system. SUMMARY
[0004] The present application provides a training method and device of an image segmentation model, an electronic device and a storage medium to solve the defects of the image segmentation model in the prior art and improve the optimization efficiency of the image segmentation model.
[0005] The application provides a training method of an image segmentation model, comprising: obtaining a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model, each first hyperparameter group comprising a value combination of all hyperparameters; the plurality of hyperparameters comprising a learning rate, a momentum, a weight decay, a warm-up period and a class weight; training the preset model based on a training sample to obtain a fitness value corresponding to the first hyperparameter group; determining a plurality of local optimal hyperparameter groups from the plurality of first hyperparameter groups based on the fitness value; the fitness value being determined based on image segmentation accuracy of the preset model and image segmentation speed; adjusting all first hyperparameter groups based on the local optimal hyperparameter groups to obtain a plurality of second hyperparameter groups, and taking the second hyperparameter group with the highest fitness value as a global optimal hyperparameter group; when there is a second hyperparameter group with a fitness value greater than a fitness threshold, adjusting all second hyperparameter groups based on the global optimal hyperparameter group to obtain a plurality of third hyperparameter groups; and taking the third hyperparameter group with the highest fitness value as a trained hyperparameter group of the preset model to obtain the image segmentation model.
[0006] According to the training method of the image segmentation model provided by the application, the plurality of local optimal hyperparameter groups are determined from the plurality of first hyperparameter groups based on the fitness value, comprising: classifying all first hyperparameter groups based on the fitness value to obtain a plurality of hyperparameter group categories; and taking the first hyperparameter group with the highest fitness value in each hyperparameter group category as a local optimal hyperparameter group.
[0007] According to the training method of the image segmentation model provided by the application, the plurality of second hyperparameter groups are obtained by adjusting all first hyperparameter groups based on the local optimal hyperparameter groups, comprising: moving each first hyperparameter group in a hyperparameter group category to the local optimal hyperparameter group of the hyperparameter group category to obtain a second hyperparameter group of the hyperparameter group category; and obtaining a plurality of second hyperparameter groups based on the second hyperparameter groups of all hyperparameter group categories.
[0008] According to the training method of the image segmentation model provided by the application, the image segmentation speed is determined based on the calculation complexity and the frame rate per unit time of the preset model, and the image segmentation accuracy is determined based on the image segmentation precision and the precision score of the preset model, and the fitness value is determined based on the following steps: training the preset model based on a training sample, determining the image segmentation precision, the precision score, the frame rate per unit time and the calculation complexity based on the image segmentation result output by the preset model; obtaining a first weight of the image segmentation precision, a second weight of the precision score, a third weight of the frame rate per unit time and a fourth weight of the calculation complexity; performing weighted summation on the image segmentation precision, the precision score and the frame rate per unit time based on the first weight, the second weight and the third weight to obtain a fitness numerator; obtaining a fitness denominator based on the fourth weight and the calculation complexity; and obtaining the fitness value based on the ratio of the fitness numerator to the fitness denominator.
[0009] The training method of the image segmentation model provided by the application comprises the following steps: obtaining a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model, wherein each first hyperparameter group comprises a value combination of all hyperparameters; the plurality of hyperparameters comprise a learning rate, a momentum, a weight decay, a warm-up period and a class weight; obtaining a plurality of historical hyperparameter groups based on historical value combinations of the plurality of hyperparameters; obtaining a plurality of random hyperparameter groups based on random value combinations of the plurality of hyperparameters; obtaining a first hyperparameter group by performing weighted summation on one historical hyperparameter group and one random hyperparameter group; and obtaining a plurality of first hyperparameter groups based on the weighted summation results of the plurality of historical hyperparameter groups and the plurality of random hyperparameter groups.
[0010] The training method of the image segmentation model provided by the application comprises the following steps: obtaining a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model, wherein each first hyperparameter group comprises a value combination of all hyperparameters; the plurality of hyperparameters comprise a learning rate, a momentum, a weight decay, a warm-up period and a class weight; obtaining a plurality of historical hyperparameter groups based on historical value combinations of the plurality of hyperparameters; obtaining a plurality of random hyperparameter groups based on random value combinations of the plurality of hyperparameters; obtaining a first hyperparameter group by performing weighted summation on one historical hyperparameter group and one random hyperparameter group; and obtaining a plurality of first hyperparameter groups based on the weighted summation results of the plurality of historical hyperparameter groups and the plurality of random hyperparameter groups.
[0011] The training method of the image segmentation model provided by the application comprises the following steps: obtaining a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model, wherein each first hyperparameter group comprises a value combination of all hyperparameters; the plurality of hyperparameters comprise a learning rate, a momentum, a weight decay, a warm-up period and a class weight; obtaining a plurality of historical hyperparameter groups based on historical value combinations of the plurality of hyperparameters; obtaining a plurality of random hyperparameter groups based on random value combinations of the plurality of hyperparameters; obtaining a first hyperparameter group by performing weighted summation on one historical hyperparameter group and one random hyperparameter group; and obtaining a plurality of first hyperparameter groups based on the weighted summation results of the plurality of historical hyperparameter groups and the plurality of random hyperparameter groups.
[0012] The training method of the image segmentation model provided by the application comprises the following steps: obtaining a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model, wherein each first hyperparameter group comprises a value combination of all hyperparameters; the plurality of hyperparameters comprise a learning rate, a momentum, a weight decay, a warm-up period and a class weight; obtaining a plurality of historical hyperparameter groups based on historical value combinations of the plurality of hyperparameters; obtaining a plurality of random hyperparameter groups based on random value combinations of the plurality of hyperparameters; obtaining a first hyperparameter group by performing weighted summation on one historical hyperparameter group and one random hyperparameter group; and obtaining a plurality of first hyperparameter groups based on the weighted summation results of the plurality of historical hyperparameter groups and the plurality of random hyperparameter groups.
[0013] The application further provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the training method of any one of the image segmentation models when executing the computer program.
[0014] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the training method of any one of the image segmentation models.
[0015] The training method of the image segmentation model, the device, the electronic device and the storage medium provided by the application realize accurate evaluation of the first hyperparameter group, the second hyperparameter group and the third hyperparameter group by calculating the fitness value, and improve the accuracy of determining the local optimal hyperparameter group and the global optimal hyperparameter group. The application optimizes all the first hyperparameter groups according to the local optimal hyperparameter group, realizes the approximation of the first hyperparameter group to the local optimal hyperparameter group, adjusts all the second hyperparameter groups according to the global optimal hyperparameter group, realizes the approximation of the second hyperparameter group to the global optimal hyperparameter group, improves the efficiency of determining the trained hyperparameter group, and further improves the training efficiency of the image segmentation model. BRIEF DESCRIPTION OF DRAWINGS
[0016] In order to more clearly illustrate the technical solutions of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.
[0017] Figure 1 is one of the flowcharts of the training method of the image segmentation model provided by the application.
[0018] Figure 2 is the second flowchart of the training method of the image segmentation model provided by the application.
[0019] Figure 3 is the structural diagram of the training device of the image segmentation model provided by the application.
[0020] Figure 4 is the structural diagram of the electronic device provided by the application. DETAILED DESCRIPTION
[0021] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.
[0022] The training method of the image segmentation model provided in the present application will be described below in conjunction with the drawings. Figures 1-4 The training method of the image segmentation model provided in the present application will be described below in conjunction with the drawings.
[0023] Figure 1 is one of the flowcharts of the training method of the image segmentation model provided in the present application, as shown in Figure 1 The training method of the image segmentation model includes steps S100 to S400, and each step is specifically as follows.
[0024] S100: Based on a plurality of value combinations of a plurality of hyperparameters of a preset model, a plurality of first hyperparameter groups are obtained.
[0025] Each first hyperparameter group includes one value combination of all hyperparameters; the plurality of hyperparameters include a learning rate, a momentum, a weight decay, a warm-up period and a class weight.
[0026] The preset model includes an improved YOLACT++ (You Only Look At Coefficients++) architecture model. For example, a Mobile Neural Network Version 3 (MobileNetV3) with a parameter amount ≤2.5 million (M). The preset model is used for image segmentation.
[0027] The hyperparameters include key parameters of the preset model, which can directly affect the convergence speed, segmentation accuracy and computing resource consumption of the preset model. For example, the plurality of hyperparameters include a learning rate, a momentum, a weight decay, a warm-up period and a class weight. The learning rate is used to control the magnitude of adjusting the parameters in the training process of the preset model along the gradient direction of the loss function. The value range of the learning rate is [0.01, 1.0]. The momentum is used to accelerate the optimization process of the preset model and reduce oscillation. The value range of the momentum is [0.6, 0.98]. The weight decay is used to prevent overfitting of the preset model. The value range of the weight decay is [0, 0.001]. The warm-up period is used to ensure the smooth growth of the learning rate. The value range of the warm-up period is [0, 5.0]. The class weight is used to adapt the preset model to different categories. The value range of the class weight is [0.1, 5.0].
[0028] The learning rate, momentum, weight decay, warm-up period, and class weight are a hyperparameter group. According to various combinations of values of the hyperparameters in the hyperparameter group, a plurality of first hyperparameter groups are obtained. For example, the first hyperparameter groups include [0.2, 0.7, 0.0005, 3.0, 2.0] and [0.2, 0.8, 0.0004, 2.0, 3.0]. Each first hyperparameter group includes a combination of values of all hyperparameters.
[0029] S200: training the preset model based on the training sample to obtain an adaptability value corresponding to the first hyperparameter group; and determining a plurality of local optimal hyperparameter groups from the plurality of first hyperparameter groups based on the adaptability value.
[0030] The adaptability value is determined based on the image segmentation accuracy of the preset model and the speed of image segmentation.
[0031] The training sample includes high-precision multispectral data of a complex farmland scene collected by a multispectral camera and a laser radar and the like. The multispectral data can provide richer spectral information to more accurately segment crops, weeds and obstacles, and improve the environmental perception ability of the agricultural machinery equipment of the autonomous vehicle. To enhance the robustness of the training sample, the present application adopts a plurality of data enhancement strategies, including dynamic light compensation and occlusion simulation, to reduce the influence of light changes, shadow interference and vegetation density changes in the complex farmland environment on the instance segmentation model. The light change compensation adopts brightness normalization and adaptive contrast enhancement technology, so that the model can maintain stable segmentation performance under different light conditions; the random occlusion enhancement simulates the target occlusion situation caused by the agricultural machinery equipment and the interlaced vegetation, to avoid the performance decline of the preset model when the local information is missing. In addition, the present application also introduces noise disturbance of different intensities, such as salt and pepper noise and Gaussian noise, into the training sample to increase the complexity of the training sample and improve the anti-interference ability of the preset model when facing environmental interference, so as to ensure the stability and accuracy of the preset model in the training sample of the complex farmland scene.
[0032] A calculation function of the adaptability value is constructed according to the image segmentation accuracy of the preset model and the speed of image segmentation.
[0033] Each first hyperparameter group is traversed to obtain a combination of values of the hyperparameter group of the preset model. After the training sample is labeled, the preset model is input for training. For each first hyperparameter group, the adaptability value is calculated according to the calculation function of the adaptability value, to obtain the adaptability value corresponding to each first hyperparameter group. According to the adaptability value corresponding to the first hyperparameter group, a plurality of local optimal hyperparameter groups are selected from all first hyperparameter groups.
[0034] S300: Based on the local optimal hyperparameter group, adjust all first hyperparameter groups to obtain multiple second hyperparameter groups, and use the second hyperparameter group with the highest fitness value as the global optimal hyperparameter group.
[0035] Based on the local optimal hyperparameter group, all first hyperparameter groups are adjusted so that the values of the hyperparameters in all first hyperparameter groups are close to the values of the hyperparameters in the corresponding local optimal hyperparameter group, thereby obtaining multiple second hyperparameter groups.
[0036] Traverse each second hyperparameter group to obtain a combination of hyperparameter values for the preset model. Input the labeled training samples into the preset model for training. For each second hyperparameter group, calculate the fitness value using the fitness value calculation function to obtain the fitness value corresponding to each second hyperparameter group. The second hyperparameter group with the highest fitness value is determined as the global optimal hyperparameter group.
[0037] S400: When there is a second hyperparameter group whose fitness value is greater than the fitness threshold, all second hyperparameter groups are adjusted based on the global optimal hyperparameter group to obtain multiple third hyperparameter groups; the third hyperparameter group with the highest fitness value is used as the trained hyperparameter group of the preset model to obtain an image segmentation model.
[0038] Based on the global optimal hyperparameter group, all second hyperparameter groups are adjusted so that the values of the hyperparameters in all second hyperparameter groups are close to the values of the hyperparameters in the global optimal hyperparameter group, thereby obtaining multiple third hyperparameter groups.
[0039] If the fitness value of any second hyperparameter group is greater than the fitness threshold (e.g., 0.5), each third hyperparameter group is traversed to obtain the value combination of the hyperparameter group for the preset model. The labeled training samples are input into the preset model for training. For each third hyperparameter group, the fitness value is calculated according to the fitness value calculation function to obtain the fitness value corresponding to each third hyperparameter group. The third hyperparameter group with the highest fitness value is used as the final trained hyperparameter group for the preset model to obtain the image segmentation model. The image segmentation model is used to segment crops, non-crops, and obstacles in farmland environment images.
[0040] The calculation formula for the third hyperparameter group is as follows.
[0041] ; in, For the A third hyperparameter group, For the The second hyperparameter group, is the global optimal hyperparameter group, is the aggregation factor, which is used to control the intensity of the aggregation of the second hyperparameter group. The value range of the is [0.1, 0.6].
[0042] When the second hyperparameter group appears a significant optimization signal (the fitness value of the second hyperparameter group is greater than the fitness threshold), the global search resource is integrated, the overall convergence of all second hyperparameter groups is promoted, and the stability of the optimization result is improved.
[0043] The training method of the image segmentation model provided by the embodiment of the application realizes accurate evaluation of the first hyperparameter group, the second hyperparameter group and the third hyperparameter group by calculating the fitness value, and improves the accuracy of determining the local optimal hyperparameter group and the global optimal hyperparameter group. The application optimizes all first hyperparameter groups according to the local optimal hyperparameter group, realizes the approximation of the first hyperparameter group to the local optimal hyperparameter group, adjusts all second hyperparameter groups according to the global optimal hyperparameter group, realizes the approximation of the second hyperparameter group to the global optimal hyperparameter group, improves the efficiency of determining the trained hyperparameter group, and further improves the training efficiency of the image segmentation model.
[0044] Optionally, as shown in Figure 2 When the fitness value of the second hyperparameter group is greater than the fitness threshold, the method further comprises the following steps of: updating the first hyperparameter group according to the third hyperparameter group; repeating steps S200 to S400, and further iteratively updating the first hyperparameter group until a preset iteration number (for example, 100 times) is reached, and ending the iteration.
[0045] The application further improves the accuracy of the final trained hyperparameter group by cyclically iterating the first hyperparameter group.
[0046] Based on the above embodiment, the speed of image segmentation is determined based on the calculation complexity of the preset model and the number of frames processed per unit time, and the accuracy of image segmentation is determined based on the image segmentation accuracy and the accuracy score of the preset model. The fitness value is determined based on the following steps: Training the preset model based on the training sample, determining the image segmentation accuracy, the accuracy score, the number of frames processed per unit time and the calculation complexity based on the image segmentation result output by the preset model; Obtaining a first weight of the image segmentation accuracy, a second weight of the accuracy score, a third weight of the number of frames processed per unit time and a fourth weight of the calculation complexity; Based on the first weight, the second weight and the third weight, the image segmentation accuracy, the accuracy score and the number of frames processed per unit time are weighted and summed to obtain the fitness numerator; Based on the fourth weight and the calculation complexity, the fitness denominator is obtained. The fitness value is obtained based on the ratio of the fitness numerator and the fitness denominator.
[0047] The formula for calculating the fitness value is as follows.
[0048] ; in, is the fitness value, is the image segmentation accuracy, is the accuracy score, is the number of frames processed per unit time, is the computational complexity, is the first weight, is the second weight, is the third weight, is the fourth weight, is the fitness numerator, is the fitness denominator.
[0049] The present invention constructs a calculation function for the fitness value according to image segmentation accuracy, accuracy score, number of frames processed per unit time and computational complexity, which is conducive to the subsequent accurate evaluation of the first hyperparameter group, the second hyperparameter group and the third hyperparameter group.
[0050] Acquiring multiple first hyperparameter groups based on multiple value combinations of multiple hyperparameters of a preset model includes the following steps: Obtain a historical hyperparameter group based on historical value combinations of multiple hyperparameters; Obtain a random hyperparameter group based on a random combination of multiple hyperparameter values; Perform weighted summation on a historical hyperparameter group and a random hyperparameter group to obtain a first hyperparameter group; A plurality of first hyperparameter groups are obtained according to weighted summation results of a plurality of historical hyperparameter groups and a plurality of random hyperparameter groups.
[0051] ; in, For the The first hyperparameter group, is the control parameter, The value of is [0, 1], For the historical hyperparameter sets, For the random hyperparameter sets, is a random disturbance.
[0052] The historical hyperparameter group includes the hyperparameter value combination with the historical optimal fitness value.
[0053] The fitness value of the current first hyperparameter group is compared with the fitness value of the historical hyperparameter group. If the fitness value of the first hyperparameter group is found to be better, the historical hyperparameter group is updated according to the first hyperparameter group.
[0054] The size of the control parameters is dynamically adjusted by changing the fitness value to ensure fast and effective convergence to the optimal first hyperparameter group.
[0055] After obtaining the optimal first set of hyperparameters, a perturbation strategy is used for local search (for example, adjusting random perturbations ), avoid falling into the trap of local optimal solution and enhance the global search capability.
[0056] The embodiment of the present invention obtains the first hyperparameter group by performing weighted summation on the historical hyperparameter group and the random hyperparameter group, which takes into account both the historical values of the hyperparameter group and the random changes of the hyperparameter group, thereby improving the accuracy of the first hyperparameter group.
[0057] Based on the above embodiment, determining multiple local optimal hyperparameter groups from multiple first hyperparameter groups based on fitness values includes the following steps: Classifying all first hyperparameter groups based on the fitness values to obtain a plurality of hyperparameter group categories; In each hyperparameter group category, the first hyperparameter group with the highest fitness value is taken as the local optimal hyperparameter group.
[0058] Based on the fitness values, all first hyperparameter groups are classified to obtain multiple hyperparameter group categories. For example, the specific steps of classifying the first hyperparameter group are as follows.
[0059] ; in, For the The first hyperparameter group, is the maximum fitness value among all the fitness values of the first hyperparameter groups, For the The fitness value of the first hyperparameter group, is the hyperparameter group category 1, is the hyperparameter group category 2, is the hyperparameter group category 3.
[0060] In each category of hyperparameter groups, the first hyperparameter group with the highest fitness value is taken as the local optimal hyperparameter group .
[0061] The present invention classifies all first hyperparameter groups according to the classification of fitness values, and then obtains the local optimal hyperparameter group, which is beneficial to improving the efficiency of optimizing each type of hyperparameter group category.
[0062] Based on the above embodiment, based on the local optimal hyperparameter group, all first hyperparameter groups are adjusted to obtain multiple second hyperparameter groups, including: moving each first hyperparameter group in a hyperparameter group category closer to the local optimal hyperparameter group of the hyperparameter group category to obtain the second hyperparameter group of the hyperparameter group category; based on the second hyperparameter groups of all hyperparameter group categories, multiple second hyperparameter groups are obtained.
[0063] Each first hyperparameter group in a hyperparameter group category is moved closer to a local optimal hyperparameter group in the hyperparameter group category to obtain a second hyperparameter group in the hyperparameter group category, including the following steps: In a hyperparameter group category, obtaining a difference value of each first hyperparameter group relative to the local optimal hyperparameter group; Based on the product value of each difference value and the random factor, a plurality of reduced difference values are obtained; A second hyperparameter set is obtained based on the sum of each reduced difference value and the corresponding first hyperparameter set.
[0064] The calculation formula for obtaining the second hyperparameter group is as follows.
[0065] ; in, For the The local optimal hyperparameter group of the class hyperparameter group, For the The first hyperparameter group, For the The second hyperparameter group, is a random factor, The value range of is [0.1, 0.3], is the difference value, To reduce the difference.
[0066] The present invention adjusts the random factor , which avoids being affected by the local optimal hyperparameter group and helps to increase the probability of finding a better second hyperparameter group.
[0067] Based on the above embodiment, after the third hyperparameter group with the highest fitness value is used as the trained hyperparameter group of the preset model and the image segmentation model is obtained, the following steps are further included: Obtain the image of the farmland to be segmented; Inputting the image to be segmented into the image segmentation model and obtaining the image segmentation results output by the image segmentation model; the image segmentation results are used to identify crops, non-crops and obstacles; Determine the driving route of agricultural machinery in the farmland based on the image segmentation results.
[0068] In order to enable the image segmentation model obtained after training to run efficiently on the embedded computing device of the agricultural machinery equipment, the inference acceleration technology is adopted in the present application. For example, quantization inference acceleration is performed using Tensor Runtime (TensorRT), the weights of the image segmentation model are compressed and optimized, the image segmentation model is adapted to the embedded computing device, and thus the inference speed is improved and the computing resource occupation is reduced. At the same time, the optimized image segmentation model is deployed to the embedded platform of the agricultural machinery equipment, for example, NVIDIA Jetson AGX Orin with advanced GPU expansion architecture, to realize real-time processing in the farmland environment.
[0069] The multispectral camera, the laser radar and the Inertial Measurement Unit (IMU) sensor are used to collect the image to be segmented in the farmland environment in real time. The image to be segmented is input into the image segmentation model to obtain the image segmentation result output by the image segmentation model.
[0070] The agricultural machinery equipment combines the image segmentation result with the path planning algorithm to dynamically adjust the running track of the agricultural machinery equipment.
[0071] The present application applies the image segmentation model to the agricultural machinery equipment, obtains the image segmentation result through the image segmentation model, and realizes automatic identification of crops, non-crops and obstacles. The driving route of the agricultural machinery equipment in the farmland is determined based on the image segmentation result, which not only improves the work efficiency, but also enhances the safety of the automatic driving of the agricultural machinery equipment.
[0072] The training device of the image segmentation model provided by the present application is described below, and the training device of the image segmentation model described below can be correspondingly referred to the training method of the image segmentation model described above.
[0073] As shown in Figure 3 An image segmentation model training device, comprising a first acquisition module 301, a first determination module 302, a second determination module 303 and a second acquisition module 304.
[0074] The first acquisition module 301 is configured to acquire a plurality of first hyperparameter groups based on a plurality of value combination of a plurality of hyperparameters of a preset model, each first hyperparameter group comprising a value combination of all hyperparameters; the plurality of hyperparameters comprising a learning rate, a momentum, a weight decay, a warm-up period and a class weight.
[0075] The first determining module 302 is configured to train a preset model based on a training sample, obtain an fitness value corresponding to a first hyperparameter group, and determine a plurality of local optimal hyperparameter groups from the plurality of first hyperparameter groups based on the fitness value. The fitness value is determined based on image segmentation accuracy of the preset model and speed of image segmentation.
[0076] The second determining module 303 is configured to adjust all the first hyperparameter groups based on the local optimal hyperparameter groups, obtain a plurality of second hyperparameter groups, and take a second hyperparameter group with the highest fitness value as a global optimal hyperparameter group.
[0077] The second obtaining module 304 is configured to adjust all the second hyperparameter groups based on the global optimal hyperparameter group when there is a second hyperparameter group with a fitness value greater than a fitness threshold, obtain a plurality of third hyperparameter groups, and take a third hyperparameter group with the highest fitness value as a trained hyperparameter group of the preset model to obtain an image segmentation model.
[0078] The training device of the image segmentation model provided by the embodiment of the present application realizes accurate evaluation of the first hyperparameter group, the second hyperparameter group, and the third hyperparameter group by calculating the fitness value, and improves the accuracy of determining the local optimal hyperparameter group and the global optimal hyperparameter group. According to the embodiment of the present application, all the first hyperparameter groups are optimized based on the local optimal hyperparameter group, the first hyperparameter group is approximated to the local optimal hyperparameter group, all the second hyperparameter groups are adjusted based on the global optimal hyperparameter group, the second hyperparameter group is approximated to the global optimal hyperparameter group, the efficiency of determining the trained hyperparameter group is improved, and the training efficiency of the image segmentation model is further improved.
[0079] In one embodiment, the first determining module 302 is configured to classify all the first hyperparameter groups based on the fitness value to obtain a plurality of hyperparameter group categories, and take a first hyperparameter group with the highest fitness value in each hyperparameter group category as a local optimal hyperparameter group.
[0080] In one embodiment, the first determining module 302 is configured to make each first hyperparameter group in a hyperparameter group category close to a local optimal hyperparameter group of the hyperparameter group category to obtain a second hyperparameter group of the hyperparameter group category, and obtain a plurality of second hyperparameter groups based on the second hyperparameter groups of all the hyperparameter group categories.
[0081] In one embodiment, the speed of image segmentation is determined based on the calculation complexity of the preset model and the frame number processed per unit time, and the image segmentation accuracy is determined based on the image segmentation precision and the precision score of the preset model. The first determination module 302 is configured to: train the preset model based on the training sample, determine the image segmentation precision, the precision score, the frame number processed per unit time, and the calculation complexity based on the image segmentation result output by the preset model; obtain a first weight of the image segmentation precision, a second weight of the precision score, a third weight of the frame number processed per unit time, and a fourth weight of the calculation complexity; perform weighted summation on the image segmentation precision, the precision score, and the frame number processed per unit time based on the first weight, the second weight, and the third weight to obtain a fitness numerator; obtain a fitness denominator based on the fourth weight and the calculation complexity; and obtain a fitness value based on the ratio of the fitness numerator to the fitness denominator.
[0082] In one embodiment, the first obtaining module 301 is configured to: obtain a plurality of historical hyperparameter groups based on a plurality of historical value combinations of the plurality of hyperparameters; obtain a plurality of random hyperparameter groups based on a plurality of random value combinations of the plurality of hyperparameters; perform weighted summation on one historical hyperparameter group and one random hyperparameter group to obtain a first hyperparameter group; and obtain a plurality of first hyperparameter groups based on the weighted summation results of the plurality of historical hyperparameter groups and the plurality of random hyperparameter groups.
[0083] In one embodiment, the first determination module 302 is configured to: in one hyperparameter group category, obtain a difference value of each first hyperparameter group relative to a local optimal hyperparameter group; obtain a plurality of reduced difference values based on the product of each difference value and a random factor; and obtain a second hyperparameter group based on the sum of each reduced difference value and the corresponding first hyperparameter group.
[0084] In one embodiment, the training device of the image segmentation model further includes an application module configured to: obtain an image to be segmented of a farmland; input the image to be segmented into the image segmentation model to obtain an image segmentation result output by the image segmentation model; the image segmentation result is used to identify crops, non-crops, and obstacles; and determine a driving route of an agricultural machinery device in the farmland based on the image segmentation result.
[0085] Figure 4 An example of a schematic diagram of the physical structure of an electronic device is shown in Figure 4As shown, the electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 complete mutual communication through the communications bus 440. The processor 410 can invoke a logical instruction in the memory 430 to execute a training method of an image segmentation model, the method including: based on a plurality of value combinations of a plurality of hyperparameters of a preset model, obtaining a plurality of first hyperparameter groups, each first hyperparameter group including one value combination of all hyperparameters; the plurality of hyperparameters including a learning rate, a momentum, a weight decay, a warm-up period, and a class weight; training the preset model based on a training sample to obtain a fitness value corresponding to the first hyperparameter group; based on the fitness value, determining a plurality of local optimal hyperparameter groups from the plurality of first hyperparameter groups; the fitness value being determined based on image segmentation accuracy of the preset model and speed of image segmentation; based on the local optimal hyperparameter group, adjusting all first hyperparameter groups to obtain a plurality of second hyperparameter groups, and taking the second hyperparameter group with the highest fitness value as a global optimal hyperparameter group; when there is a second hyperparameter group with a fitness value greater than a fitness threshold, adjusting all second hyperparameter groups based on the global optimal hyperparameter group to obtain a plurality of third hyperparameter groups; and taking the third hyperparameter group with the highest fitness value as a trained hyperparameter group of the preset model to obtain the image segmentation model.
[0086] In addition, the logical instructions in the memory 430 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.
[0087] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the training method of the image segmentation model provided by the above method, and the method comprises: obtaining a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model, each first hyperparameter group comprising a value combination of all hyperparameters; the plurality of hyperparameters comprise a learning rate, a momentum, a weight decay, a warm-up period and a class weight; training the preset model based on a training sample to obtain a fitness value corresponding to the first hyperparameter group; determining a plurality of local optimal hyperparameter groups from the plurality of first hyperparameter groups based on the fitness value; the fitness value is determined based on the image segmentation accuracy of the preset model and the speed of image segmentation; adjusting all first hyperparameter groups based on the local optimal hyperparameter groups to obtain a plurality of second hyperparameter groups, and taking the second hyperparameter group with the highest fitness value as a global optimal hyperparameter group; when there is a second hyperparameter group with a fitness value greater than a fitness threshold, adjusting all second hyperparameter groups based on the global optimal hyperparameter group to obtain a plurality of third hyperparameter groups; and taking the third hyperparameter group with the highest fitness value as the trained hyperparameter group of the preset model to obtain the image segmentation model.
[0088] The device embodiments described above are only schematic, wherein the units shown as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., may be located in one place or distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.
[0089] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus necessary universal hardware platforms, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in terms of the contribution to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.
[0090] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A training method for an image segmentation model, characterized in that: include: Based on multiple value combinations of multiple hyperparameters of a preset model, obtaining multiple first hyperparameter groups, each of which includes a value combination of all the hyperparameters; the multiple hyperparameters include a learning rate, momentum, weight decay, a warm-up period, and a class weight; Training the preset model based on the training sample to obtain a fitness value corresponding to the first hyperparameter group; Based on the fitness value, determining a plurality of local optimal hyperparameter groups from a plurality of the first hyperparameter groups; The fitness value is determined based on the image segmentation accuracy and image segmentation speed of the preset model; Based on the local optimal hyperparameter group, adjust all the first hyperparameter groups to obtain multiple second hyperparameter groups, and take the second hyperparameter group with the highest fitness value as the global optimal hyperparameter group; When there is a second hyperparameter group whose fitness value is greater than the fitness threshold, all the second hyperparameter groups are adjusted based on the global optimal hyperparameter group to obtain multiple third hyperparameter groups; the third hyperparameter group with the highest fitness value is used as the trained hyperparameter group of the preset model to obtain an image segmentation model.
2. The image segmentation model training method according to claim 1, characterized in that: The step of determining a plurality of local optimal hyperparameter groups from a plurality of the first hyperparameter groups based on the fitness value comprises: Classifying all of the first hyperparameter groups based on the fitness values to obtain a plurality of hyperparameter group categories; In each of the hyperparameter group categories, the first hyperparameter group with the highest fitness value is used as the local optimal hyperparameter group.
3. The image segmentation model training method according to claim 2, characterized in that: The adjusting all the first hyperparameter groups based on the local optimal hyperparameter group to obtain multiple second hyperparameter groups includes: Each first hyperparameter group in one of the hyperparameter group categories is moved closer to a local optimal hyperparameter group in the hyperparameter group category to obtain a second hyperparameter group in the hyperparameter group category; Based on the second hyperparameter groups of all the hyperparameter group categories, a plurality of second hyperparameter groups are obtained.
4. The image segmentation model training method according to claim 1, characterized in that: The speed of image segmentation is determined based on the computational complexity of the preset model and the number of frames processed per unit time. The accuracy of image segmentation is determined based on the image segmentation precision and accuracy score of the preset model. The fitness value is determined based on the following steps: Training the preset model based on the training samples, and determining the image segmentation accuracy, the accuracy score, the number of frames processed per unit time, and the computational complexity based on the image segmentation result output by the preset model; Obtaining a first weight of the image segmentation accuracy, a second weight of the accuracy score, a third weight of the number of frames processed per unit time, and a fourth weight of the computational complexity; Based on the first weight, the second weight, and the third weight, performing a weighted summation on the image segmentation accuracy, the accuracy score, and the number of frames processed per unit time to obtain a fitness numerator; Obtaining a fitness denominator based on the fourth weight and the computational complexity; The fitness value is obtained based on the ratio of the fitness numerator to the fitness denominator.
5. The image segmentation model training method according to claim 1, characterized in that: The method of obtaining a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model includes: Obtaining a historical hyperparameter group based on a combination of historical values of a plurality of the hyperparameters; Obtaining a random hyperparameter group based on a random value combination of a plurality of the hyperparameters; Performing weighted summation on the historical hyperparameter group and the random hyperparameter group to obtain the first hyperparameter group; A plurality of first hyperparameter groups are obtained according to a weighted summation result of a plurality of the historical hyperparameter groups and a plurality of the random hyperparameter groups.
6. The image segmentation model training method according to claim 3, characterized in that: The step of moving each first hyperparameter group in a hyperparameter group category closer to a local optimal hyperparameter group of the hyperparameter group category to obtain a second hyperparameter group of the hyperparameter group category includes: In one of the hyperparameter group categories, obtaining a difference value of each of the first hyperparameter groups relative to the local optimal hyperparameter group; obtaining a plurality of reduced difference values based on a product value of each of the difference values and a random factor; The second hyperparameter set is obtained based on the sum of each of the reduced difference values and the corresponding first hyperparameter set.
7. The image segmentation model training method according to claim 1, characterized in that: After the image segmentation model is obtained by using the third hyperparameter group with the highest fitness value as the trained hyperparameter group of the preset model, the method further includes: Obtain the image of the farmland to be segmented; Inputting the image to be segmented into the image segmentation model, and obtaining an image segmentation result output by the image segmentation model; wherein the image segmentation result is used to identify crops, non-crops, and obstacles; The driving route of the agricultural machinery in the farmland is determined based on the image segmentation result.
8. A training device for an image segmentation model, characterized in that: include: A first acquisition module is configured to acquire a plurality of first hyperparameter groups based on a plurality of value combinations of a plurality of hyperparameters of a preset model, each of the first hyperparameter groups including a value combination of all the hyperparameters; the plurality of hyperparameters including a learning rate, momentum, weight decay, a warm-up period, and a class weight; A first determination module is used to train the preset model based on the training sample and obtain a fitness value corresponding to the first hyperparameter group; Based on the fitness value, determining a plurality of local optimal hyperparameter groups from a plurality of the first hyperparameter groups; The fitness value is determined based on the image segmentation accuracy and image segmentation speed of the preset model; A second determining module is configured to adjust all of the first hyperparameter groups based on the local optimal hyperparameter group to obtain multiple second hyperparameter groups, and to select the second hyperparameter group with the highest fitness value as the global optimal hyperparameter group; The second acquisition module is used to adjust all the second hyperparameter groups based on the global optimal hyperparameter group to obtain multiple third hyperparameter groups when the fitness value of the second hyperparameter group is greater than the fitness threshold; and use the third hyperparameter group with the highest fitness value as the trained hyperparameter group of the preset model to obtain an image segmentation model.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the training method of the image segmentation model according to any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the training method of the image segmentation model according to any one of claims 1 to 7 is implemented.