A model training method and device, equipment and storage medium

Abstract

A model training method and device, equipment and storage medium are disclosed. The method comprises: obtaining a preset number of image samples in a training sample set; obtaining a first image and a target coordinate; scaling the preset number of image samples according to the target coordinate, and pasting the scaled image samples into the first image according to the target coordinate to obtain a target image sample; and training a first model through the target image sample to obtain a target model. Through the technical scheme of the present application, the model training efficiency can be improved while the accuracy of the model is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence, and in particular to a model training method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, pedestrian intelligent labeling technology based on video images has become possible.

[0003] In the prior art, a first model is trained based on a training sample set. Since the number of image samples in the training sample set is large, the training time is long. Therefore, a plurality of image samples are randomly cropped and spliced into a new image sample, and the first model is trained based on a training sample set formed by the new image sample after splicing. Since the new image sample is obtained by randomly cropping a plurality of image samples, the pedestrian data in the image sample may be missing, which further leads to insufficient model learning and low model accuracy. SUMMARY

[0004] Embodiments of the present application provide a model training method, device, equipment and storage medium, which solve the problem that since the new image sample is obtained by randomly cropping a plurality of image samples, the pedestrian data in the image sample may be missing, which further leads to insufficient model learning and low model accuracy.

[0005] According to an aspect of the present application, a model training method is provided, comprising:

[0006] acquiring a preset number of image samples in a training sample set;

[0007] acquiring a first image and a target coordinate;

[0008] scaling the preset number of image samples according to the target coordinate, and pasting the scaled image samples into the first image according to the target coordinate to obtain a target image sample;

[0009] training a first model based on the target image sample to obtain a target model.

[0010] According to another aspect of the present application, a model training device is provided, comprising:

[0011] a first acquisition module configured to acquire a preset number of image samples in a training sample set;

[0012] a second acquisition module configured to acquire a first image and a target coordinate;

[0013] The first obtaining module is configured to scale the preset number of image samples according to the target coordinates, and paste the scaled image samples into a first image according to the target coordinates to obtain target image samples;

[0014] The second obtaining module is configured to train a first model by using the target image samples to obtain a target model.

[0015] According to another aspect of the present application, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory connected with the at least one processor; wherein,

[0018] The memory stores a computer program which can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the model training method according to any one of the embodiments of the present application.

[0019] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the model training method according to any one of the embodiments of the present application when executed.

[0020] The embodiments of the present application obtain a preset number of image samples in a training sample set, obtain a first image and target coordinates, scale the preset number of image samples according to the target coordinates, paste the scaled image samples into the first image according to the target coordinates to obtain target image samples, and train a first model by using the target image samples to obtain a target model, thereby solving the problem of insufficient model learning and low model accuracy due to the fact that new image samples are obtained by randomly cropping multiple image samples, and the pedestrian data in the image samples is missing.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0023] Figure 1 is a flowchart of a model training method in embodiment one of the present application;

[0024] Figure 2 is a schematic diagram of a second image in embodiment one of the present application;

[0025] Figure 3 is a schematic diagram of a first model in embodiment one of the present application;

[0026] Figure 4 is a schematic diagram of a generated feature map in embodiment one of the present application;

[0027] Figure 5 is a structural schematic diagram of a model training device in embodiment two of the present application;

[0028] Figure 6 is a structural schematic diagram of an electronic device in embodiment three of the present application. DETAILED DESCRIPTION

[0029] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in accordance with relevant laws and regulations.

[0032] Embodiment one

[0033] Figure 1 is a flowchart of a model training method in Embodiment One of the present application. The present embodiment can be applied to the case of pedestrian detection. The method can be performed by a model training device in an embodiment of the present application. The device can be implemented in software and / or hardware. As shown in FIG. 1, the method specifically includes the following steps: Figure 1

[0034] S110, obtaining a preset number of image samples in a training sample set.

[0035] The training sample set can be a sample set required for training the to-be-trained model. The training sample set can include image samples. The preset number of image samples can be preset according to user requirements. For example, four image samples in the training sample set can be obtained.

[0036] S120, obtaining a first image and target coordinates.

[0037] The first image can be a randomly generated picture. It should be noted that the size of the first image needs to be greater than the size of the picture to be input into the to-be-trained model. For example, the first image can be a gray 3-channel picture with a length of h0 and a width of w0. The picture to be input into the to-be-trained model has a length of h and a width of w, h0 is 1.5 times h, and w0 is 1.5 times w. i i i i

[0038] The target coordinates can be coordinates randomly generated according to the size of the first image. They are the intersection points after processing all the preset number of image samples. For example, if the preset number of image samples is four pictures, the first image has a length of h0 and a width of w0. A coordinate (x, y) is randomly generated in the range of (0.2w0~0.8w0, 0.2h0~0.8h0) of the first image. The coordinate is the coordinate of the right lower corner, the left lower corner, the right upper corner, and the left upper corner of the four pictures after scaling. c c

[0039] S130, scaling the preset number of image samples according to the target coordinates, and pasting the scaled image samples into the first image according to the target coordinates to obtain a target image sample.

[0040] The target image sample is obtained by reasonably pasting the preset number of image samples after scaling to the corresponding positions of the first image, and then scaling the first image to the required picture size of the to-be-trained model. The synthesized new image is the target image sample.

[0041] ​​​​​​​Specifically, the manner of scaling the preset number of image samples according to the target coordinate and pasting the scaled image samples into the first image according to the target coordinate to obtain the target image sample can be: generating a target coordinate at random, scaling the preset number of image samples according to the target coordinate, pasting the scaled image samples into corresponding positions in the first image, and then scaling the first image with the pasted image samples to a picture size required by the to-be-trained model to synthesize the target image sample.

[0042] Optionally, the scaling the preset number of image samples according to the target coordinate and pasting the scaled image samples into the first image according to the target coordinate to obtain the target image sample includes:

[0043] determining a scaling ratio corresponding to the image sample according to the target coordinate and the size of the image sample;

[0044] scaling the image sample according to the scaling ratio corresponding to the image sample to obtain a scaled image sample;

[0045] determining a pasting position of the preset number of scaled image samples according to the target coordinate;

[0046] pasting the preset number of scaled image samples into the first image according to the pasting position of the preset number of scaled image samples to obtain a second image;

[0047] scaling the second image to obtain the target image sample.

[0048] The scaling ratio is a ratio of a size of the pasting position of the image sample formed according to the target coordinate to the size of the image sample. The second image is an image in which all the preset number of scaled image samples are pasted in the first image. It should be noted that, because the pasting position of each image sample is different, that is, the pasting size is different, the corresponding scaling ratio is also different.

[0049] Specifically, the manner of determining the scaling ratio corresponding to the image sample according to the target coordinate and the size of the image sample can be: determining a size of the scaled image sample according to the target coordinate, and determining the ratio of the determined size of the scaled image sample to the size of the image sample as the scaling ratio.

[0050] Specifically, the manner of scaling the image sample according to the scaling ratio corresponding to the image sample to obtain the scaled image sample can be: after all the image samples are calculated to obtain the corresponding scaling ratio, the image sample is scaled according to the corresponding scaling ratio to obtain all the preset number of scaled image samples.

[0051] Specifically, the method for determining the pasting position of a preset number of scaled image samples based on the target coordinates can be as follows: In the first image, the pasting position of each scaled image sample is divided according to the target coordinates and the preset number of scaled image samples.

[0052] Specifically, the method to obtain the second image by pasting a preset number of scaled image samples into the first image according to the pasting positions of the preset number of scaled image samples can be as follows: paste the preset number of scaled image samples into the corresponding positions of the first image, and determine the image in the first image with the preset number of scaled image samples pasted in it as the second image.

[0053] Specifically, the method for scaling the second image to obtain the target image sample can be as follows: scale the second image to the image size required in the model to be trained to obtain the target image sample.

[0054] In a specific example Figure 2 This is a schematic diagram of a second image in Embodiment 1 of the present invention, as shown below. Figure 2 As shown, four image samples of a preset quantity of four are selected. First, a grayscale 3-channel image with a length of h0 and a width of w0 is generated, where h0 and w0 are the image sizes required in the model to be trained. i w i 1.5 times; in the generated grayscale 3-channel image, a coordinate (x) is randomly generated within the range of (0.2w0~0.8w0, 0.2h0~0.8h0). c y c These coordinates represent the coordinates of the bottom right, bottom left, top right, and top left corners of the four scaled image samples, respectively. The four image samples are scaled along their longest sides. Taking the first image sample as an example, if the dimensions of the first image sample are h and w, the dimension of the pasting position in the grayscale 3-channel image is y. c x c If the scaling is done along the longer side, the scaling ratio is r = min(y). c / h,x c / w), the scaled image size is y c ·r、x c It should be noted that the scaling ratios of the remaining three image samples are calculated in a similar way to those of the first image sample, and will not be repeated here. After calculating the scaling ratios of the four image samples, the four image samples are scaled according to their respective scaling ratios, and then the scaled image samples are pasted into... Figure 2 The second image is synthesized by taking the positions opposite the four image samples in the grayscale 3-channel image shown; finally, the second image is scaled to the required image size h in the model to be trained. i, w i , to obtain a target image sample.

[0055] S140, training the first model by the target image sample to obtain a target model.

[0056] The first model is a to-be-trained model, which can be an improved network model, for example, can be a yolov5 network model. Figure 3 is a schematic diagram of a first model in Embodiment One of the present application, as shown in Figure 3 The first model can be an improved yolov5 network model, which is to improve the three-layer network structure containing FPN+PAN in the original yolov5 network model into a four-layer network structure. Figure 3 The backbone in the improved yolov5 network model is mainly used to extract features in the image sample for use in the subsequent network, and the neck is used to better utilize the features in the image sample extracted by the backbone to obtain a prediction box. The improved yolov5 network model can better extract information in the target image sample in the training sample set, and can fuse the strong semantic features (beneficial to classification) at the top layer and the high-resolution information (beneficial to positioning) at the bottom layer. The target model is a trained model obtained by training the first model according to the target image sample.

[0057] Specifically, the way to train the first model by the target image sample to obtain the target model can be: determining the first model and the target image sample, iteratively training the first model according to the target image sample, and obtaining the target model.

[0058] Optionally, training the first model by the target image sample to obtain the target model comprises:

[0059] generating a target image sample set according to at least one target image sample;

[0060] inputting a target image in the target image sample set into the first model to obtain a prediction box;

[0061] training parameters of the first model according to a target function generated according to the prediction box and a target box corresponding to the preset number of image samples;

[0062] returning to the operation of inputting the target image in the target image sample set into the first model to obtain the prediction box until the target model is obtained.

[0063] The target image sample set can be a sample set generated according to at least one synthesized target image sample, and is also a sample set for training the first model. The prediction box can be an outline box of a pedestrian to be collected in the target image obtained by inputting the target image into the first model. The target box is an outline box of a pedestrian to be collected in the preset number of image samples.

[0064] Optionally, the target function comprises: a bounding box coordinate loss function, a confidence loss function, and a prediction probability loss function, the confidence loss function is

[0065] wherein S is the width of the feature map or the height of the feature map, SxS is the number of grids, B is the number of anchor boxes, C is the confidence, is the kth output feature map, the mth cell, and the nth anchor box, if it is a positive sample, it is 1, otherwise it is 0, is the kth output feature map, the mth cell, and the nth anchor box, if it is a negative sample, it is 1, otherwise it is 0, is the kth output feature map, the mth cell, and the nth anchor box, if the GIOU of the ignore region is greater than 0.5, it is 0, otherwise it is 1.

[0066] wherein the target function can be:

[0067]

[0068] wherein GIoUloss is the loss of coordinate regression of the target box and the predicted box, the bounding box coordinate loss function is the prediction probability loss function is wherein K is the number of output feature maps, c is the class, classes is the total number of classes, p m (c) is the probability of predicting the cth class.

[0069] wherein the ignore region can be a separate pedestrian region in the target image, and can be a dense crowd region in the target image.

[0070] It should be noted that the coefficient in the target function can filter the predicted box whose GIOU of the ignore region is greater than 0.5, so that it does not participate in the loss function calculation.

[0071] Optionally, the target image in the target image sample set is input into the first model to obtain a predicted box, comprising:

[0072] down-sampling the target image sample to obtain a first feature map of a first size;

[0073] down-sampling the first feature map of the first size to obtain a first feature map of a second size;

[0074] downsample the first feature map of the second size to obtain a first feature map of a third size;

[0075] downsample the first feature map of the third size to obtain a first feature map of a fourth size;

[0076] convolve the first feature map of the fourth size to obtain a second feature map of the fourth size;

[0077] upsample the second feature map of the fourth size to obtain a second feature map of the third size;

[0078] generate a third feature map of the third size according to the a1, the b1, the second feature map of the third size and the first feature map of the third size;

[0079] upsample the third feature map of the third size to obtain a second feature map of the second size;

[0080] generate a third feature map of the second size according to the a2, the b2, the second feature map of the second size and the first feature map of the second size;

[0081] upsample the third feature map of the second size to obtain a second feature map of the first size;

[0082] generate a third feature map of the first size according to the a3, the b3, the second feature map of the first size and the first feature map of the first size;

[0083] convolve the third feature map of the first size to obtain a fourth feature map of the first size;

[0084] downsample the fourth feature map of the first size to obtain a fourth feature map of the second size;

[0085] generate a fifth feature map of the second size according to the a4, the b4, the fourth feature map of the second size and the third feature map of the second size;

[0086] downsample the fifth feature map of the second size to obtain a fourth feature map of the third size;

[0087] generate a fifth feature map of the third size according to the a5, the b5, the fourth feature map of the third size and the third feature map of the third size;

[0088] downsample the fifth feature map of the third size to obtain a third feature map of the fourth size;

[0089] generate a fourth feature map of the fourth size according to the a6, the b6, the third feature map of the fourth size and the second feature map of the fourth size;

[0090] Input the fourth feature map of the first size, the fifth feature map of the second size, the fifth feature map of the third size and the fourth feature map of the fourth size into convolution layers respectively to obtain a prediction frame of the target image sample.

[0091] Down-sampling can be referred to as down-sampling, mainly for reducing the image; up-sampling can be referred to as image interpolation, mainly for enlarging the image, so that it can be displayed on a display device with higher resolution. The principle of down-sampling can be: for an image with a size of M*N, down-sampling it by f times, that is, obtaining a resolution image with a size of (M / f)*(N / f). It should be noted that f should be the greatest common divisor of M and N. The principle of up-sampling can be: using an interpolation method, that is, inserting new elements between pixel points on the basis of the original image pixels using a suitable interpolation algorithm, thereby enlarging the image. It should be noted that the interpolation algorithm includes traditional interpolation and edge-based interpolation, and also includes region-based image interpolation.

[0092] The first feature map, the second feature map, the third feature map, the fourth feature map and the fifth feature map are only for distinguishing the feature maps in the process of obtaining the prediction frame of the target image sample, and have no other special meanings. The first size, the second size, the third size and the fourth size respectively represent different sizes in the process of obtaining the prediction frame of the target image sample. For example, the first size can be 152*152, the second size can be 76*76, the third size can be 38*38 and the fourth size can be 19*19.

[0093] The convolution can be a convolution calculation, that is, a convolution operation on the feature map, that is, feature extraction on the feature map to generate a new feature map.

[0094] The alpha1, beta1, alpha2, beta2, alpha3, beta3, alpha4, beta4, alpha5, beta5, alpha6 and beta6 are all adaptive fusion factors.

[0095] Specifically, the way of generating the third feature map of the third size according to the alpha1, the beta1, the second feature map of the third size and the first feature map of the third size can be: adding the alpha1*the second feature map of the third size to the beta1*the first feature map of the third size to obtain the third feature map of the third size.

[0096] Specifically, the way of generating the third feature map of the second size according to the alpha2, the beta2, the second feature map of the second size and the first feature map of the second size can be: adding the alpha2*the second feature map of the second size to the beta2*the first feature map of the second size to obtain the third feature map of the second size.

[0097] Specifically, the manner of generating the third feature map of the first size according to the alpha 3, the beta 3, the second feature map of the first size and the first feature map of the first size can be that the third feature map of the first size can be obtained by adding the alpha 3*the second feature map of the first size and the beta 3*the first feature map of the first size.

[0098] Specifically, the manner of generating the fifth feature map of the second size according to the alpha 4, the beta 4, the fourth feature map of the second size and the third feature map of the second size can be that the fifth feature map of the second size can be obtained by adding the alpha 4*the fourth feature map of the second size and the beta 4*the third feature map of the second size.

[0099] Specifically, the manner of generating the fifth feature map of the third size according to the alpha 5, the beta 5, the fourth feature map of the third size and the third feature map of the third size can be that the fifth feature map of the third size can be obtained by adding the alpha 5*the fourth feature map of the third size and the beta 5*the third feature map of the third size.

[0100] Specifically, the manner of generating the fourth feature map of the fourth size according to the alpha 6, the beta 6, the third feature map of the fourth size and the second feature map of the fourth size can be that the fourth feature map of the fourth size can be obtained by adding the alpha 6*the third feature map of the fourth size and the beta 6*the second feature map of the fourth size.

[0101] In one specific example, Figure 4 is a schematic diagram of generating a feature map in the embodiment one of the present application, such as Figure 4As shown, the target image sample is 608*608*3, the first size is 152*152, the second size is 76*76, the third size is 38*38, and the fourth size is 19*19, the upward arrow can represent down-sampling, and the downward arrow can represent up-sampling. After down-sampling the target image sample, a first feature map of 152*152 is obtained; after down-sampling the first feature map of 152*152, a first feature map of 76*76 is obtained; after down-sampling the first feature map of 76*76, a first feature map of 38*38 is obtained; after down-sampling the first feature map of 38*38, a first feature map of 19*19 is obtained; after convolution calculation on the first feature map of 19*19, a second feature map of 19*19 is obtained; after up-sampling the second feature map of 19*19, a second feature map of 38*38 is obtained; adding the second feature map of 38*38 of a1*38*38 to the first feature map of 38*38 of b1*38*38 can obtain a third feature map of 38*38; after up-sampling the third feature map of 38*38, a second feature map of 76*76 is obtained; adding the second feature map of 76*76 of a2*76*76 to the first feature map of 76*76 of b2*76*76 can obtain a third feature map of 76*76; after up-sampling the third feature map of 76*76, a second feature map of 152*152 is obtained; adding the second feature map of 152*152 of a3*152*152 to the first feature map of 152*152 of b3*152*152 can obtain a third feature map of 152*152; after convolution calculation on the third feature map of 152*152, a fourth feature map of 152*152 is obtained; after down-sampling the fourth feature map of 152*152, a fourth feature map of 76*76 is obtained; adding the fourth feature map of 76*76 of a4*76*76 to the third feature map of 76*76 of b4*76*76 can obtain a fifth feature map of 76*76; after down-sampling the fifth feature map of 76*76, a fourth feature map of 38*38 is obtained; adding the fourth feature map of 38*38 of a5*38*38 to the third feature map of 38*38 of b5*38*38 can obtain a fifth feature map of 38*38; after down-sampling the fifth feature map of 38*38, a third feature map of 19*19 is obtained; adding the third feature map of 19*19 of a6*19*19 to the second feature map of 19*19 of b6*19*19 can obtain a fourth feature map of 19*19; the fourth feature map of 152*152, the fifth feature map of 76*76, the fifth feature map of 38*38, and the fourth feature map of 19*19 are respectively input into a convolution layer to obtain a prediction box of the target image sample.

[0102] In the process of obtaining the prediction box of the target image sample, the FPN+PAN structure of yolov5 in the prior art is improved from the original three-layer structure to a four-layer structure, and the added one layer structure can better detect smaller targets. Meanwhile, an adaptive fusion factor is introduced to fuse two feature maps, which can obtain features with stronger expression ability.

[0103] Optionally, further comprising:

[0104] The first feature map of the fourth size is up-sampled into a second feature map of a third size, and a first vector is obtained through global average pooling;

[0105] The first feature map of the third size is subjected to 1*1 convolution to become a feature map of the same channel number as the second feature map of the third size, and a second vector is obtained through global average pooling; Figure 1

[0106] The first vector and the second vector are used to determine a1 and b1.

[0107] The global average pooling is a method of assigning a feature map to each category, adding all pixel values in the feature map to obtain a numerical value, and using the numerical value to represent the corresponding feature map as the input of the subsequent softmax layer, so as to replace the fully connected layer, reduce the number of parameters, reduce the calculation amount, and reduce overfitting.

[0108] The first vector and the second vector are both vectors obtained through global average pooling of the feature map.

[0109] Specifically, the way of determining a1 and b1 according to the first vector and the second vector can be that position elements corresponding to the first vector and the second vector are subjected to softmax activation to obtain weight values a1 and b1 between 0 and 1.

[0110] It should be noted that the calculation methods of a2, b2, a3, b3, a4, b4, a5, b5, a6 and b6 are similar to those of a1 and b1, which will not be described here.

[0111] In a specific example, a1 and b1 are adaptive fusion factors, the third size is 38*38, and the fourth size is 19*19. j j The first feature map of 19*19 is up-sampled into a second feature map F1 of 38*38, and a vector x1 is obtained through global average pooling, the length of which is consistent with the channel number of the F1 feature map. The first feature map of 38*38 is subjected to 1*1 convolution to become a feature map P1 of the same channel number as the second feature map F1 of 38*38, and a vector y1 is obtained through global average pooling, the length of which is consistent with that of x1. Position elements corresponding to the vector x1 and the vector y1 are subjected to softmax activation to obtain weight values a1 and b1 between 0 and 1, as shown in the following formula:

[0112] ​​

[0113]

[0114] Wherein, i is the number of channels, alpha 1i , beta 1i respectively represent the respective weights of the feature maps F1, P1 when fused on the i channel, and the fused feature map Y1 is:

[0115] Y1 = alpha1F1 + beta1P1;

[0116] The rest of alpha j and beta j are calculated in the same way, which can be learned in depth during neural network model training, and by adaptively fusing the two feature maps, each channel can be fused according to the weight, and a feature with stronger expression capability can be obtained.

[0117] The technical scheme of the embodiment, by acquiring a preset number of image samples in a training sample set; acquiring a first image and a target coordinate; scaling the preset number of image samples according to the target coordinate, and pasting the scaled image samples into the first image according to the target coordinate to obtain a target image sample; training a first model through the target image sample to obtain a target model, solves the problem that the new image sample is obtained by randomly cropping multiple image samples, which may result in missing pedestrian data in the image sample, and further leads to insufficient model learning and low model accuracy, and can improve the model training efficiency while improving the model accuracy.

[0118] Embodiment two

[0119] Figure 5 is a structural schematic diagram of a model training device in embodiment two of the application. The embodiment can be applicable to the case of pedestrian detection, and the device can be realized in the form of software and / or hardware, and can be integrated in any device providing model training function, such as Figure 5 As shown in the figure, the model training device specifically comprises: a first acquisition module 210, a second acquisition module 220, a first obtaining module 230 and a first obtaining module 240.

[0120] The first acquisition module 210 is configured to acquire a preset number of image samples in a training sample set.

[0121] The second acquisition module 220 is configured to acquire a first image and a target coordinate.

[0122] The first obtaining module 230 is configured to scale the preset number of image samples according to the target coordinate, and paste the scaled image samples into the first image according to the target coordinate to obtain a target image sample.

[0123] The second obtaining module 240 is configured to obtain a target model by training a first model using the target image sample.

[0124] Optionally, the first obtaining module is specifically configured to:

[0125] determine a scaling ratio corresponding to the image sample according to the target coordinate and a size of the image sample;

[0126] scale the image sample according to the scaling ratio corresponding to the image sample to obtain a scaled image sample;

[0127] determine a pasting position of the preset number of scaled image samples according to the target coordinate;

[0128] paste the preset number of scaled image samples into a first image according to the pasting position of the preset number of scaled image samples to obtain a second image;

[0129] scale the second image to obtain a target image sample.

[0130] Optionally, the second obtaining module is specifically configured to:

[0131] generate a target image sample set according to at least one target image sample;

[0132] input a target image in the target image sample set into a first model to obtain a prediction box;

[0133] train parameters of the first model according to a target function generated according to the prediction box and a target box corresponding to the preset number of image samples;

[0134] return to perform the operation of inputting the target image in the target image sample set into the first model to obtain the prediction box until a target model is obtained.

[0135] Optionally, the second obtaining module is specifically configured to:

[0136] down-sample the target image sample to obtain a first feature map of a first size;

[0137] down-sample the first feature map of the first size to obtain a first feature map of a second size;

[0138] down-sample the first feature map of the second size to obtain a first feature map of a third size;

[0139] down-sample the first feature map of the third size to obtain a first feature map of a fourth size;

[0140] perform convolution on the first feature map of the fourth size to obtain a second feature map of the fourth size;

[0141] upsample the second feature map of the fourth size to obtain a second feature map of a third size;

[0142] generate a third feature map of the third size according to the al, the bl, the second feature map of the third size, and the first feature map of the third size;

[0143] upsample the third feature map of the third size to obtain a second feature map of a second size;

[0144] generate a third feature map of the second size according to the a2, the b2, the second feature map of the second size, and the first feature map of the second size;

[0145] upsample the third feature map of the second size to obtain a second feature map of a first size;

[0146] generate a third feature map of the first size according to the a3, the b3, the second feature map of the first size, and the first feature map of the first size;

[0147] convolve the third feature map of the first size to obtain a fourth feature map of the first size;

[0148] downsample the fourth feature map of the first size to obtain a fourth feature map of a second size;

[0149] generate a fifth feature map of the second size according to the a4, the b4, the fourth feature map of the second size, and the third feature map of the second size;

[0150] downsample the fifth feature map of the second size to obtain a fourth feature map of a third size;

[0151] generate a fifth feature map of the third size according to the a5, the b5, the fourth feature map of the third size, and the third feature map of the third size;

[0152] downsample the fifth feature map of the third size to obtain a third feature map of a fourth size;

[0153] generate a fourth feature map of the fourth size according to the a6, the b6, the third feature map of the fourth size, and the second feature map of the fourth size;

[0154] input the fourth feature map of the first size, the fifth feature map of the second size, the fifth feature map of the third size, and the fourth feature map of the fourth size into convolution layers respectively to obtain a prediction box of the target image sample.

[0155] Optionally, the second obtaining module is further configured to:

[0156] obtain a first vector obtained by upsampling the first feature map of the fourth size to the second feature map of the third size and then performing global average pooling.

[0157] The first feature map of the third size is subjected to 1*1 convolution and becomes a second feature map of the third size. Figure 1 The second vector is obtained by performing global average pooling on the feature map of the third size.

[0158] Alpha1 and beta1 are determined according to the first vector and the second vector.

[0159] Optionally, the target function comprises a bounding box coordinate loss function, a confidence loss function and a prediction probability loss function, and the confidence loss function is

[0160] wherein S is the width of the feature map or the height of the feature map, S*S is the number of grids, B is the number of anchor boxes, C is the confidence, is whether the kth output feature map, the mth cell and the nth anchor box are positive samples, and is 1 if they are positive samples, and is 0 otherwise, is whether the kth output feature map, the mth cell and the nth anchor box are negative samples, and is 1 if they are negative samples, and is 0 otherwise, is whether the kth output feature map, the mth cell and the nth anchor box have a GIOU greater than 0.5 with the ignore region, and is 0 if the GIOU is greater than 0.5, and is 1 otherwise.

[0161] The product can execute the method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method.

[0162] The technical scheme of the embodiment, by obtaining a preset number of image samples in a training sample set; obtaining a first image and target coordinates; scaling the preset number of image samples according to the target coordinates, and pasting the scaled image samples into the first image according to the target coordinates to obtain target image samples; training a first model through the target image samples to obtain a target model, solves the problem that since new image samples are obtained by randomly cropping multiple image samples, the pedestrian data in the image samples is missing, which further leads to insufficient model learning and low model accuracy, and improves the model training efficiency and the model accuracy.

[0163] Embodiment three

[0164] Figure 6This is a schematic diagram of an electronic device according to Embodiment 3 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0165] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0166] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0167] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as model training methods.

[0168] In some embodiments, the model training method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, portions or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the model training method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the model training method by other means, e.g., with the aid of firmware.

[0169] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0170] Computer programs used to implement the methods of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0171] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0173] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0174] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0175] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0176] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A model training method, characterized in that, The method comprises the following steps: obtaining a preset number of image samples in a training sample set; obtaining a first image and a target coordinate; scaling the preset number of image samples according to the target coordinate, and pasting the scaled image samples into the first image according to the target coordinate to obtain a target image sample; training a first model through the target image sample to obtain a target model; wherein training the first model through the target image sample to obtain the target model comprises: generating a target image sample set according to at least one target image sample; inputting a target image in the target image sample set into the first model to obtain a prediction box; training parameters of the first model according to a target function generated by the prediction box and a target box corresponding to the preset number of image samples; returning to the operation of inputting the target image in the target image sample set into the first model to obtain the prediction box until the target model is obtained; wherein inputting the target image in the target image sample set into the first model to obtain the prediction box comprises: down-sampling the target image sample to obtain a first feature map of a first size; down-sampling the first feature map of the first size to obtain a first feature map of a second size; down-sampling the first feature map of the second size to obtain a first feature map of a third size; down-sampling the first feature map of the third size to obtain a first feature map of a fourth size; convolving the first feature map of the fourth size to obtain a second feature map of the fourth size; up-sampling the second feature map of the fourth size to obtain a second feature map of the third size; generating a third feature map of the third size according to α1, β1, the second feature map of the third size and the first feature map of the third size; up-sampling the third feature map of the third size to obtain a second feature map of the second size; generating a third feature map of the second size according to α2, β2, the second feature map of the second size and the first feature map of the second size; up-sampling the third feature map of the second size to obtain a second feature map of the first size; generating a third feature map of the first size according to α3, β3, the second feature map of the first size and the first feature map of the first size; convolving the third feature map of the first size to obtain a fourth feature map of the first size; down-sampling the fourth feature map of the first size to obtain a fourth feature map of the second size; generating a fifth feature map of the second size according to α4, β4, the fourth feature map of the second size and the third feature map of the second size; down-sampling the fifth feature map of the second size to obtain a fourth feature map of the third size; generating a fifth feature map of the third size according to α5, β5, the fourth feature map of the third size and the third feature map of the third size; down-sampling the fifth feature map of the third size to obtain a third feature map of the fourth size; generating a fourth feature map of the fourth size according to α6, β6, the third feature map of the fourth size and the second feature map of the fourth size; input the fourth feature map of the first size, the fifth feature map of the second size, the fifth feature map of the third size and the fourth feature map of the fourth size into convolution layers respectively to obtain a prediction box of the target image sample; wherein, α1, β1, α2, β2, α3, β3, α4, β4, α5, β5, α6 and β6 are adaptive fusion factors; The determination mode of the α1 and the β1 is: obtaining a first vector obtained by performing global average pooling on the fourth size first feature map after up-sampling into a third size second feature map; obtaining a second vector obtained by performing global average pooling on a feature map with the same channel number as the third size second feature map after performing 1*1 convolution on the third size first feature map; determining α1 and β1 according to the first vector and the second vector.

2. The method of claim 1, wherein, scaling the preset number of image samples according to the target coordinates, and pasting the scaled image samples into a first image according to the target coordinates to obtain target image samples, comprising: determining a scaling ratio corresponding to the image sample according to the target coordinates and the size of the image sample; scaling the image sample according to the scaling ratio corresponding to the image sample to obtain a scaled image sample; determining the pasting positions of the preset number of scaled image samples according to the target coordinates; pasting the preset number of scaled images into the first image according to the pasting positions of the preset number of scaled image samples to obtain a second image; scaling the second image to obtain a target image sample.

3. The method of claim 1, wherein, The target function comprises: a box coordinate loss function, a confidence loss function and a prediction probability loss function, and the confidence loss function is wherein S is the width or height of the feature map, S*S is the number of grids, B is the number of anchor boxes, and C is the confidence, is the kth output feature map, the mth cell, and the nth anchor box is a positive sample, if it is a positive sample, it is 1, otherwise it is 0, is the kth output feature map, the mth cell, and the nth anchor box is a negative sample, if it is a negative sample, it is 1, otherwise it is 0, is the kth output feature map, the mth cell, and the nth anchor box and the ignore region GIOU is greater than 0.5, if it is greater than 0.5, it is 0, otherwise it is 1.

4. A model training apparatus characterized by comprising: comprising: The first obtaining module is configured to obtain a preset number of image samples in a training sample set. The second obtaining module is configured to obtain a first image and target coordinates. The first obtaining module is configured to obtain a preset number of image samples in a training sample set. The second obtaining module is configured to train the first model by using the target image samples to obtain a target model. The second obtaining module is configured to: generate a target image sample set according to at least one target image sample; input a target image in the target image sample set into the first model to obtain a prediction box; train parameters of the first model according to a target function generated according to the prediction box and a target box corresponding to the preset number of image samples; return to execute the operation of inputting the target image in the target image sample set into the first model to obtain the prediction box until the target model is obtained; The operation of inputting the target image in the target image sample set into the first model to obtain the prediction box comprises: down-sampling the target image sample to obtain a first feature map of a first size; down-sampling the first feature map of the first size to obtain a first feature map of a second size; down-sampling the first feature map of the second size to obtain a first feature map of a third size; downsample the first feature map of the third size to obtain a first feature map of a fourth size; perform convolution on the first feature map of the fourth size to obtain a second feature map of the fourth size; upsample the second feature map of the fourth size to obtain a second feature map of a third size; generate a third feature map of the third size according to the first feature map of the third size, the second feature map of the third size, the first feature map of the fourth size, the alpha 1, and the beta 1; upsample the third feature map of the third size to obtain a second feature map of a second size; generate a third feature map of the second size according to the second feature map of the second size, the first feature map of the second size, the alpha 2, and the beta 2; upsample the third feature map of the second size to obtain a second feature map of a first size; generate a third feature map of the first size according to the second feature map of the first size, the first feature map of the first size, the alpha 3, and the beta 3; perform convolution on the third feature map of the first size to obtain a fourth feature map of the first size; downsample the fourth feature map of the first size to obtain a fourth feature map of a second size; generate a fifth feature map of the second size according to the fourth feature map of the second size, the third feature map of the second size, the alpha 4, and the beta 4; downsample the fifth feature map of the second size to obtain a fourth feature map of a third size; generate a fifth feature map of the third size according to the fourth feature map of the third size, the third feature map of the third size, the alpha 5, and the beta 5; downsample the fifth feature map of the third size to obtain a third feature map of a fourth size; generate a fourth feature map of the fourth size according to the third feature map of the fourth size, the second feature map of the fourth size, the alpha 6, and the beta 6; input the fourth feature map of the first size, the fifth feature map of the second size, the fifth feature map of the third size, and the fourth feature map of the fourth size into convolution layers respectively to obtain a prediction box of a target image sample; the alpha 1, the beta 1, the alpha 2, the beta 2, the alpha 3, the beta 3, the alpha 4, the beta 4, the alpha 5, the beta 5, the alpha 6, and the beta 6 are adaptive fusion factors; the alpha 1 and the beta 1 are determined in the following manner: a first vector is obtained by performing global average pooling on the second feature map of the third size which is obtained by upsampling the first feature map of the fourth size; a second vector is obtained by performing global average pooling on a feature map of the third size which is obtained by performing 1*1 convolution on the first feature map of the third size; the alpha 1 and the beta 1 are determined according to the first vector and the second vector.

5. The apparatus of claim 4, wherein, the first obtaining module is specifically configured to: determine a scaling ratio corresponding to the image sample according to the target coordinate and the size of the image sample; scale the image sample according to the scaling ratio corresponding to the image sample to obtain a scaled image sample; determine a pasting position of the preset number of scaled image samples according to the target coordinate; paste the preset number of scaled image samples into the first image according to the pasting position of the preset number of scaled image samples to obtain a second image; scale the second image to obtain a target image sample.

6. An electronic device, comprising: the electronic device comprises: at least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the model training method in any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a processor to implement the model training method in any one of claims 1-3 when executed.

Images