Counting model training method and device
By constructing coordinate attention network and head network, combining residual network and progressive feature map pyramid network, the problem of low crop counting accuracy is solved and the accuracy of crop counting is improved.
Patent Information
- Application Number
- CN202411139376.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-12-31
AI Technical Summary
In the prior art, crop counting accuracy is low, especially when crops are densely planted, and existing counting algorithms or models are difficult to accurately identify and count.
Build a coordinate attention network and a head network, combine residual network and progressive feature map pyramid network, build a counting model, process training images through the backbone network, acquire feature maps, and calculate prediction frames and category confidence through the head network, based on loss Optimize model parameters.
It improves the accuracy of crop counting and solves the problem of low crop counting accuracy.
Smart Images
Figure CN119229275A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, and in particular to a training method and device for a counting model. Background Art
[0002] At present, the most common counting applications are crowd counting and animal counting. In these application scenarios, the objects that need to be counted are highly independent or have obvious differences. The existing counting algorithms or counting models are also targeted at these objects. In agricultural production, it is often necessary to count crops within a unit to understand the emergence rate, which is of great significance to increasing the output value of crops. Because crops are generally planted densely and each crop is blocked by another, if the existing counting algorithm or counting model is used directly, the accuracy will inevitably be low. Summary of the invention
[0003] In view of this, the embodiments of the present application provide a counting model training method, device, electronic device and computer-readable storage medium to solve the problem of low crop counting accuracy in the prior art.
[0004] According to a first aspect of an embodiment of the present application, a training method for a counting model is provided, including: constructing a coordinate attention network and a head network, constructing a backbone network using the coordinate attention network and the residual network, and constructing a counting model using the backbone network, a progressive feature map pyramid network, and a head network; obtaining a training image of a target object, and inputting the training image into the counting model: processing the training image through the backbone network to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map; processing the first feature map, the second feature map, the third feature map, and the fourth feature map through a progressive feature map pyramid network to obtain a fifth feature map, a sixth feature map, a seventh feature map, and an eighth feature map; processing the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map through the head network to obtain a predicted box of the target object on the training image and a category confidence of the predicted box; calculating a counting loss based on the predicted box, the category confidence, the label of the training image, and the true box; and optimizing the model parameters of the counting model according to the counting loss to complete the training of the counting model.
[0005] According to a second aspect of an embodiment of the present application, a training device for a counting model is provided, comprising: a construction module configured to construct a coordinate attention network and a head network, use the coordinate attention network and the residual network to construct a backbone network, and use the backbone network, a progressive feature map pyramid network and the head network to construct a counting model; a first processing module configured to obtain a training image about a target object, input the training image into the counting model: process the training image through the backbone network to obtain a first feature map, a second feature map, a third feature map and a fourth feature map; a second processing module configured to use a progressive feature map pyramid network to obtain a first feature map, a second feature map, a third feature map and a fourth feature map. The network processes the first feature map, the second feature map, the third feature map and the fourth feature map to obtain the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map; the third processing module is configured to process the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map through the head network to obtain the predicted box of the target object on the training image and the category confidence of the predicted box; the calculation module is configured to calculate the counting loss based on the predicted box, the category confidence, the label of the training image and the true box; the optimization module is configured to optimize the model parameters of the counting model according to the counting loss to complete the training of the counting model.
[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0008] Compared with the prior art, the embodiments of the present application have the following beneficial effects: because the embodiments of the present application construct a coordinate attention network and a head network, use the coordinate attention network and the residual network to construct a backbone network, and use the backbone network, a progressive feature map pyramid network and a head network to construct a counting model; obtain a training image about the target object, and input the training image into the counting model: process the training image through the backbone network to obtain the first feature map, the second feature map, the third feature map and the fourth feature map; process the first feature map, the second feature map, the third feature map and the fourth feature map through the progressive feature map pyramid network to obtain the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map; process the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map through the head network to obtain the prediction box of the target object on the training image and the category confidence of the prediction box; calculate the counting loss based on the prediction box, the category confidence, the label of the training image and the true box; optimize the model parameters of the counting model according to the counting loss to complete the training of the counting model. The above technical means can solve the problem of low accuracy of crop counting in the prior art, thereby improving the accuracy of crop counting. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0010] Figure 1 It is a flowchart of a training method for a counting model provided in an embodiment of the present application;
[0011] Figure 2 is a schematic diagram of a counting method provided in an embodiment of the present application;
[0012] Figure 3 It is a structural schematic diagram of a counting model training device provided in an embodiment of the present application;
[0013] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0014] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0015] A counting model training method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.
[0016] Figure 1 It is a flowchart of a training method for a counting model provided in an embodiment of the present application. Figure 1 The counting model training method can be executed by a computer or a server, or software on the computer or the server. Figure 1 As shown, the training method of the counting model includes:
[0017] S101, build a coordinate attention network and a head network, use the coordinate attention network and the residual network to build a backbone network, and use the backbone network, the progressive feature map pyramid network and the head network to build a counting model;
[0018] S102, obtaining a training image of the target object, and inputting the training image into a counting model: processing the training image through a backbone network to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map;
[0019] S103, processing the first feature map, the second feature map, the third feature map, and the fourth feature map through a progressive feature map pyramid network to obtain a fifth feature map, a sixth feature map, a seventh feature map, and an eighth feature map;
[0020] S104, processing the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map through the head network to obtain a prediction box of the target object on the training image and a category confidence of the prediction box;
[0021] S105, calculating the counting loss based on the predicted box, the category confidence, the label of the training image and the true box;
[0022] S106, optimizing the model parameters of the counting model according to the counting loss to complete the training of the counting model.
[0023] For example, the residual network is ResNet50, which includes a zero-stage network Stage0, a first-stage network Stage1, a second-stage network Stage2, a third-stage network Stage3, and a fourth-stage network Stage4. Because the zero-stage network is some relatively simple preprocessing, the zero-stage network and the first-stage network can be divided into a stage network, so sometimes the residual model can be divided into four stage networks. AFPN (Asymptotic Feature Pyramid Network) is a feature pyramid network used in target detection tasks, which aims to solve the problem of feature information loss or degradation in traditional feature pyramid networks and strengthen direct interactions between non-adjacent layers. The residual network and the progressive feature map pyramid network are existing networks, and the coordinate attention network and the head network are self-constructed by the embodiments of the present application.
[0024] The labels of the training images are used to mark the categories of each object on the training images (to determine whether an object is the target object). The true box is used to mark the location of each object on the training images. The predicted box corresponds to the true box, and the predicted box and category confidence are finally predicted by the counting model. The category confidence of the predicted box indicates the probability that the object in the predicted box is the target object.
[0025] According to the technical solution provided by the embodiment of the present application, a coordinate attention network and a head network are constructed, a backbone network is constructed using the coordinate attention network and the residual network, and a counting model is constructed using the backbone network, a progressive feature map pyramid network and a head network; a training image about the target object is obtained, and the training image is input into the counting model: the training image is processed by the backbone network to obtain the first feature map, the second feature map, the third feature map and the fourth feature map; the first feature map, the second feature map, the third feature map and the fourth feature map are processed by the progressive feature map pyramid network to obtain the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map; the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map are processed by the head network to obtain the prediction box of the target object on the training image and the category confidence of the prediction box; the counting loss is calculated based on the prediction box, the category confidence, the label of the training image and the true box; the model parameters of the counting model are optimized according to the counting loss to complete the training of the counting model. The above technical means can solve the problem of low accuracy of crop counting in the prior art, thereby improving the accuracy of crop counting.
[0026] Furthermore, a coordinate attention network and a head network are constructed, including: using a first average pooling layer, a second average pooling layer, a feature concatenation layer, a preset convolution layer, a normalization layer, a first activation layer, a segmentation layer, a preset convolution layer, a second activation layer and a multiplication layer to construct a coordinate attention network; using an addition layer, a fully connected layer and a third activation layer to construct a head network.
[0027] The first average pooling layer performs average pooling on the feature map in the length direction, and the second average pooling layer performs average pooling on the feature map in the width direction. The feature map includes three dimensions, namely the number of channels, length and width. The preset convolution layer is a convolution layer that uses a preset convolution kernel, such as a 1*1 convolution kernel. The preset convolution layer is used to compress the number of channels of the feature map. The first activation layer can use the Hard-Swish activation function, and the second activation layer can use the Sigmoid function. The segmentation layer is used to segment a feature map into feature maps in two directions (such as vertical and horizontal directions). The multiplication layer corresponds to the multiplication operation, and the addition layer corresponds to the addition operation. Other network layers are all existing network layers.
[0028] Furthermore, the input of the coordinate attention network is recorded as the first target feature map. Inside the coordinate attention network: the first target feature map is averagely pooled in the length direction through the first average pooling layer to obtain the first pooling feature map, and the target feature map is averagely pooled in the width direction through the second average pooling layer to obtain the second pooling feature map; the first pooling feature map and the second pooling feature map are spliced through the feature splicing layer to obtain the splicing feature map; the number of channels of the splicing feature map is compressed through the preset convolution layer to obtain the convolution feature map; the convolution feature map is processed through the normalization layer, the first activation layer and the segmentation layer in turn to obtain the first segmentation feature map and the second segmentation feature map; the first segmentation feature map and the second segmentation feature map are processed through the preset convolution layer and the second activation layer in turn to obtain the first activation feature map and the second activation feature map; the first target feature map, the first activation feature map and the second activation feature map are multiplied in turn through the multiplication layer to obtain the second target feature map, wherein the second target feature map is the output of the coordinate attention network.
[0029] The normalization layer and the first activation layer are used to encode the spatial information of the convolution feature map in the vertical and horizontal directions. The segmentation layer segments the feature map output by the first activation layer into a first segmentation feature map and a second segmentation feature map in the vertical and horizontal directions.
[0030] Furthermore, a backbone network is constructed using the coordinate attention network and the residual network, including: connecting the coordinate attention network after each stage network of the residual network to obtain a backbone network; or connecting the coordinate attention network after each layer network of the residual network to obtain a backbone network; the residual network contains multiple stage networks, and each stage network contains multiple layers of networks.
[0031] The backbone network is obtained by connecting a coordinate attention network to each stage network of the residual network. The training image is processed by the backbone network, and the first stage network, the second stage network, the third stage network and the fourth stage network of the residual network process the training image to obtain the first stage feature, the second stage feature, the third stage feature and the fourth stage feature. Taking the first stage network as an example, the first target feature map input by the coordinate attention network connected to the first stage network is the first stage feature. The coordinate attention network processes the first stage feature, and the second target feature map obtained is the first feature map. The second feature map, the third feature map and the fourth feature map are similar to the first feature map.
[0032] The backbone network is obtained by connecting the coordinate attention network to each layer of the residual network. The output of each layer is used as the first target feature map of the coordinate attention network, and the second target feature map processed by the coordinate attention network is used as the new output of the layer.
[0033] In the above two methods of constructing the backbone network, the order of processing feature maps between each layer of the residual network and between each stage of the network remains unchanged.
[0034] Furthermore, the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map are processed by the head network to obtain the prediction box and the category confidence of the target object on the training image, including: inside the head network: adding the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map according to preset weights through the addition layer to obtain a comprehensive feature map; processing the comprehensive feature map through the fully connected layer and the third activation layer in turn to obtain the prediction box and the category confidence.
[0035] In the prior art, the conventional head network is composed of a fully connected layer and an activation layer. The embodiment of the present application adds an addition layer according to the structure of the backbone network and the progressive feature map pyramid network. The third activation layer can be a softmax function.
[0036] Furthermore, based on the predicted box, the category confidence, the label of the training image and the true box, the counting loss is calculated, including: calculating the intersection-over-union ratio between the predicted box and the true box; calculating the first parameter based on the label and the intersection-over-union ratio; processing the category confidence through the activation function corresponding to the second activation layer to obtain the second parameter; and calculating the counting loss based on the first parameter and the second parameter.
[0037] The counting loss L can be calculated according to the following loss function:
[0038] L=-|y-σ| β ((1-y)log(1-σ)+ylog(σ));
[0039] y is the first parameter, y is obtained by multiplying the label and the intersection and union ratio, σ is the second parameter, the category confidence can be obtained by sigmoid calculation, β is the preset parameter, || is the absolute value, and log is the logarithmic function.
[0040] IOU (Intersection over Union) is an indicator used in computer vision to evaluate the degree of overlap between the predicted box (or segmented area) and the true box (or true segmented area) in tasks such as target detection and image segmentation. It measures the degree of overlap between two areas and is a commonly used evaluation criterion in target detection algorithms.
[0041] Figure 2 Schematic diagram of a counting method provided in an embodiment of the present application. Figure 2 As shown, the method includes:
[0042] S201, obtaining an image to be detected about the target object, and inputting the image to be detected into a counting model:
[0043] S202, processing the image to be detected through the backbone network to obtain a ninth feature map, a tenth feature map, an eleventh feature map, and a twelfth feature map;
[0044] S203, processing the ninth feature map, the tenth feature map, the eleventh feature map, and the twelfth feature map through a progressive feature map pyramid network to obtain a thirteenth feature map, a fourteenth feature map, a fifteenth feature map, and a sixteenth feature map;
[0045] S204, processing the thirteenth feature map, the fourteenth feature map, the fifteenth feature map, and the sixteenth feature map through the head network to obtain a prediction box of the target object on the image to be detected and a category confidence of the prediction box;
[0046] S205 , counting the number of prediction boxes whose category confidence is greater than a preset confidence, and obtaining the number of target objects on the image to be detected.
[0047] The embodiment of the present application is the reasoning stage of the model (the stage of formally using the model after the training stage). Before the embodiment of the present application, it was the training stage of the model. The reasoning stage of the model and the training stage of the model correspond to each other, so the reasoning stage of the model will not be repeated here.
[0048] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0049] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0050] Figure 3 Schematic diagram of a counting model training device provided in an embodiment of the present application. Figure 3 As shown, the training device of the counting model includes:
[0051] A construction module 301 is configured to construct a coordinate attention network and a head network, construct a backbone network using the coordinate attention network and the residual network, and construct a counting model using the backbone network, the progressive feature map pyramid network, and the head network;
[0052] The first processing module 302 is configured to obtain a training image about the target object, input the training image into the counting model, and process the training image through the backbone network to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map;
[0053] The second processing module 303 is configured to process the first feature map, the second feature map, the third feature map and the fourth feature map through a progressive feature map pyramid network to obtain a fifth feature map, a sixth feature map, a seventh feature map and an eighth feature map;
[0054] The third processing module 304 is configured to process the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map through the head network to obtain a prediction box of the target object on the training image and a category confidence of the prediction box;
[0055] The calculation module 305 is configured to calculate the counting loss based on the predicted box, the category confidence, the label of the training image and the true box;
[0056] The optimization module 306 is configured to optimize the model parameters of the counting model according to the counting loss to complete the training of the counting model.
[0057] According to the technical solution provided by the embodiment of the present application, a coordinate attention network and a head network are constructed, a backbone network is constructed using the coordinate attention network and the residual network, and a counting model is constructed using the backbone network, a progressive feature map pyramid network and a head network; a training image about the target object is obtained, and the training image is input into the counting model: the training image is processed by the backbone network to obtain the first feature map, the second feature map, the third feature map and the fourth feature map; the first feature map, the second feature map, the third feature map and the fourth feature map are processed by the progressive feature map pyramid network to obtain the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map; the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map are processed by the head network to obtain the prediction box of the target object on the training image and the category confidence of the prediction box; the counting loss is calculated based on the prediction box, the category confidence, the label of the training image and the true box; the model parameters of the counting model are optimized according to the counting loss to complete the training of the counting model. The above technical means can solve the problem of low accuracy of crop counting in the prior art, thereby improving the accuracy of crop counting.
[0058] In some embodiments, the construction module 301 is also configured to construct a coordinate attention network using a first average pooling layer, a second average pooling layer, a feature splicing layer, a preset convolution layer, a normalization layer, a first activation layer, a segmentation layer, a preset convolution layer, a second activation layer and a multiplication layer; and to construct a head network using an addition layer, a fully connected layer and a third activation layer.
[0059] In some embodiments, the construction module 301 is also configured to record the input of the coordinate attention network as the first target feature map, and inside the coordinate attention network: the first target feature map is average pooled in the length direction through the first average pooling layer to obtain the first pooling feature map, and the target feature map is average pooled in the width direction through the second average pooling layer to obtain the second pooling feature map; the first pooling feature map and the second pooling feature map are spliced through the feature splicing layer to obtain the spliced feature map; the number of channels of the spliced feature map is compressed through the preset convolution layer to obtain the convolution feature map; the convolution feature map is processed through the normalization layer, the first activation layer and the segmentation layer in turn to obtain the first segmentation feature map and the second segmentation feature map; the first segmentation feature map and the second segmentation feature map are processed through the preset convolution layer and the second activation layer in turn to obtain the first activation feature map and the second activation feature map; the first target feature map, the first activation feature map and the second activation feature map are multiplied in turn through the multiplication layer to obtain the second target feature map, wherein the second target feature map is the output of the coordinate attention network.
[0060] In some embodiments, the construction module 301 is also configured to connect a coordinate attention network after each stage network of the residual network to obtain a backbone network; or to connect a coordinate attention network after each layer network of the residual network to obtain a backbone network; the residual network includes multiple stage networks, and each stage network includes multiple layers of networks.
[0061] In some embodiments, the third module 304 is also configured to: inside the head network, add the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map according to preset weights through the addition layer to obtain a comprehensive feature map; process the comprehensive feature map through the fully connected layer and the third activation layer in sequence to obtain a prediction box and category confidence.
[0062] In some embodiments, the calculation module 305 is further configured to calculate the intersection-over-union ratio between the predicted box and the true box; calculate the first parameter based on the label and the intersection-over-union ratio; process the category confidence through the activation function corresponding to the second activation layer to obtain the second parameter; and calculate the count loss based on the first parameter and the second parameter.
[0063] In some embodiments, the calculation module 305 is further configured to calculate the counting loss L according to the following loss function:
[0064] L=-|y-σ| β ((1-y)log(1-σ)+ylog(σ));
[0065] y is the first parameter, y is obtained by multiplying the label and the intersection and union ratio, σ is the second parameter, the category confidence can be obtained by sigmoid calculation, β is the preset parameter, || is the absolute value, and log is the logarithmic function.
[0066] In some embodiments, the optimization module 306 is further configured to obtain an image to be detected about the target object, and input the image to be detected into a counting model: process the image to be detected through the backbone network to obtain a ninth feature map, a tenth feature map, an eleventh feature map and a twelfth feature map; process the ninth feature map, the tenth feature map, the eleventh feature map and the twelfth feature map through a progressive feature map pyramid network to obtain a thirteenth feature map, a fourteenth feature map, a fifteenth feature map and a sixteenth feature map; process the thirteenth feature map, the fourteenth feature map, the fifteenth feature map and the sixteenth feature map through the head network to obtain a prediction box of the target object on the image to be detected and a category confidence of the prediction box; count the number of prediction boxes whose category confidence is greater than a preset confidence to obtain the number of target objects on the image to be detected.
[0067] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0068] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.
[0069] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.
[0070] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0071] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.
[0072] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.
[0073] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.
[0074] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical feature diagrams therein. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A training method for a counting model, characterized in that: include: Constructing a coordinate attention network and a head network, using the coordinate attention network and the residual network to construct a backbone network, and using the backbone network, a progressive feature map pyramid network, and the head network to construct a counting model; Obtain a training image of the target object and input the training image into the counting model: Processing the training image through the backbone network to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map; Processing the first feature map, the second feature map, the third feature map, and the fourth feature map through the progressive feature map pyramid network to obtain a fifth feature map, a sixth feature map, a seventh feature map, and an eighth feature map; Processing the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map through the head network to obtain a prediction box of the target object on the training image and a category confidence of the prediction box; Calculate the count loss based on the predicted box, the category confidence, the label of the training image and the true box; The model parameters of the counting model are optimized according to the counting loss to complete the training of the counting model.
2. The method according to claim 1, characterized in that: Construct the coordinate attention network and head network, including: Construct a coordinate attention network using the first average pooling layer, the second average pooling layer, the feature concatenation layer, the preset convolution layer, the normalization layer, the first activation layer, the segmentation layer, the preset convolution layer, the second activation layer, and the multiplication layer; The head network is constructed using an addition layer, a fully connected layer, and a third activation layer.
3. The method according to claim 1, characterized in that Construct a coordinate attention network, including: The input of the coordinate attention network is recorded as the first target feature map. Inside the coordinate attention network: Performing average pooling on the first target feature map in the length direction through a first average pooling layer to obtain a first pooling feature map, and performing average pooling on the target feature map in the width direction through a second average pooling layer to obtain a second pooling feature map; Splicing the first pooled feature map and the second pooled feature map through a feature splicing layer to obtain a spliced feature map; Compressing the number of channels of the concatenated feature map by a preset convolution layer to obtain a convolution feature map; Processing the convolution feature map through a normalization layer, a first activation layer, and a segmentation layer in sequence to obtain a first segmentation feature map and a second segmentation feature map; Processing the first segmentation feature map and the second segmentation feature map through a preset convolution layer and a second activation layer in sequence to obtain a first activation feature map and a second activation feature map; The first target feature map, the first activation feature map and the second activation feature map are multiplied in sequence through a multiplication layer to obtain a second target feature map, wherein the second target feature map is the output of the coordinate attention network.
4. The method according to claim 1, characterized in that The coordinate attention network and residual network are used to construct a backbone network, including: The coordinate attention network is connected after each stage network of the residual network to obtain the backbone network; or The coordinate attention network is connected after each layer of the residual network to obtain the backbone network; The residual network includes multiple stage networks, and each stage network includes a multi-layer network.
5. The method according to claim 1, characterized in that Processing the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map through the head network to obtain a prediction box of the target object on the training image and a category confidence of the prediction box includes: Inside the head network: Adding the fifth feature map, the sixth feature map, the seventh feature map and the eighth feature map according to preset weights through an addition layer to obtain a comprehensive feature map; The comprehensive feature map is processed sequentially through a fully connected layer and a third activation layer to obtain the prediction box and the category confidence.
6. The method according to claim 1, characterized in that Based on the predicted box, the category confidence, the label of the training image and the true box, the counting loss is calculated, including: Calculate the intersection-over-union ratio between the predicted frame and the true frame; Calculate a first parameter based on the label and the intersection-over-union ratio; Processing the category confidence by an activation function corresponding to the second activation layer to obtain a second parameter; The count loss is calculated based on the first parameter and the second parameter.
7. The method according to claim 1, characterized in that After optimizing the model parameters of the counting model according to the counting loss to complete the training of the counting model, the method further includes: Obtain an image to be detected of the target object, and input the image to be detected into the counting model: Processing the image to be detected through the backbone network to obtain a ninth feature map, a tenth feature map, an eleventh feature map, and a twelfth feature map; Processing the ninth feature map, the tenth feature map, the eleventh feature map, and the twelfth feature map through the progressive feature map pyramid network to obtain a thirteenth feature map, a fourteenth feature map, a fifteenth feature map, and a sixteenth feature map; Processing the thirteenth feature map, the fourteenth feature map, the fifteenth feature map and the sixteenth feature map through the head network to obtain a prediction box of the target object on the image to be detected and a category confidence of the prediction box; The number of prediction boxes whose category confidence is greater than a preset confidence is counted to obtain the number of target objects on the image to be detected.
8. A counting model training device, characterized in that: include: A construction module is configured to construct a coordinate attention network and a head network, construct a backbone network using the coordinate attention network and the residual network, and construct a counting model using the backbone network, a progressive feature map pyramid network, and the head network; A first processing module is configured to obtain a training image about the target object, input the training image into the counting model, and process the training image through the backbone network to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map; A second processing module is configured to process the first feature map, the second feature map, the third feature map and the fourth feature map through the progressive feature map pyramid network to obtain a fifth feature map, a sixth feature map, a seventh feature map and an eighth feature map; a third processing module, configured to process the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map through the head network to obtain a prediction box of the target object on the training image and a category confidence of the prediction box; A calculation module, configured to calculate a counting loss based on the predicted box, the category confidence, the label of the training image, and the true box; The optimization module is configured to optimize the model parameters of the counting model according to the counting loss to complete the training of the counting model.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.