Classification network and object detection model construction method, device, equipment and medium
By performing paired input and parameter updates on feature images of different image distribution domains, the problem of poor object detection accuracy in deep neural networks in variable environments is solved, and the accuracy of object detection in different states is improved.
Patent Information
- Application Number
- CN202011453575.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-11
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-12-11
AI Technical Summary
Existing object detection technology is difficult to achieve accurate detection under different environmental conditions and shooting styles, and the classification results of deep neural networks vary greatly under changing conditions.
By pairing input of feature images of different image distribution domains, network parameters are calculated and updated to reduce the gap in classification results, an object detection model is formed that is suitable for different states.
It improves the accuracy of object detection under different shooting states and enhances the model's detection ability of different shooting states.
Smart Images

Figure CN112613539B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer processing technology, and in particular to a classification network and object detection model construction method, device, equipment and medium. Background Art
[0002] Object detection is a core technology of computer vision, which aims to locate each object in the image and give the correct object category. Object detection, as a basic visual algorithm, has a wide range of applications. At present, object detection is generally based on deep neural networks.
[0003] However, there is an obvious problem with current object detection, that is, when the detected object is in different environmental conditions or photographed in different styles, such as day / night / foggy weather, or photographed in oil painting, sketch, etc., there will be a large difference in the object detection of the same object. Therefore, the deep neural network in the related technology cannot accurately detect objects under changing environmental conditions. Summary of the invention
[0004] In view of the above problems, a classification network and object detection model construction method, device, equipment and medium are proposed in embodiments of the present invention to overcome the above problems or at least partially solve the above problems.
[0005] In order to solve the above problems, the first aspect of the present invention discloses a classification network construction method, the method comprising:
[0006] Extracting features from multiple images of the sample object respectively to obtain corresponding multiple feature images, wherein the multiple images have different image distribution domains;
[0007] Inputting feature images corresponding to each two images with different image distribution domains into a preset network in pairs to obtain a plurality of first classification results and a plurality of second classification results;
[0008] updating the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects;
[0009] The preset network after multiple updates is determined as a classified network.
[0010] Optionally, updating the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects includes:
[0011] Determining a target loss value according to the distances between the plurality of first classification results and the plurality of second classification results;
[0012] Determine a first classification loss value according to the plurality of first classification results and the category label of the sample object, and determine a second classification loss value according to the plurality of second classification results and the category label of the sample object;
[0013] The parameters of the preset network are updated according to the target loss value, the first classification loss value and the second classification loss value.
[0014] Optionally, the method further comprises:
[0015] Processing the plurality of first classification results to determine a first result center, and processing the plurality of second classification results to determine a second result center;
[0016] Determining a target loss value according to the distances between the plurality of first classification results and the plurality of second classification results includes:
[0017] A target loss value is determined according to the gap between the first result center and the second result center.
[0018] Optionally, determining a target loss value according to a gap between the first result center and the second result center includes:
[0019] The target loss value is determined according to the following formula:
[0020]
[0021] Among them, Loss cosine is the target loss value, is the vector representation of the center of the first classification result, and is the vector representation of the center of the second classification result.
[0022] A second aspect of an embodiment of the present invention provides a method for constructing an object detection model, the method comprising:
[0023] Obtaining an original object detection model, wherein the original object detection model includes an original classification subnetwork and an original regression subnetwork;
[0024] The original classification subnetwork is replaced with the classification network described in the first aspect embodiment to obtain a target object detection model.
[0025] A third aspect of an embodiment of the present invention provides a method for constructing an object detection model, the method comprising:
[0026] Acquire a plurality of images of the sample object, wherein the plurality of images have different image distribution domains;
[0027] Inputting two images with different image distribution domains into a preset network in pairs to obtain a plurality of first object detection results and a plurality of second object detection results;
[0028] Comparing the plurality of first object detection results with the plurality of second object detection results to determine a first target object detection result and a second target object detection result representing the same object category;
[0029] Update the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and category label of the sample object;
[0030] The preset network after multiple updates is determined as the object detection model.
[0031] Optionally, updating the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and the category label of the sample object includes:
[0032] determining a target loss value according to a distance between the first target object detection result and the second target object detection result;
[0033] Determine a first object detection loss value according to the plurality of first object detection results and the position labels and the category labels of the sample objects, and determine a second object detection loss value according to the plurality of second object detection results and the position labels and the category labels of the sample objects;
[0034] The parameters of the preset network are updated according to the target loss value, the first object detection loss value, and the second object detection loss value.
[0035] Optionally, after obtaining the object detection model, the method further includes:
[0036] Obtaining an image to be detected, wherein the image to be detected has any image distribution domain among the different image distribution domains;
[0037] The image to be detected is input into the object detection model to obtain the position and category of the object contained in the image to be processed.
[0038] According to a fourth aspect of an embodiment of the present invention, a classification network construction device is provided, the device comprising:
[0039] A feature extraction module, used to extract features from multiple images of the sample object respectively to obtain corresponding multiple feature images, wherein the multiple images have different image distribution domains;
[0040] A classification result acquisition module, used for inputting the feature images corresponding to each two images with different image distribution domains into a preset network in pairs to obtain a plurality of first classification results and a plurality of second classification results;
[0041] An updating module, configured to update the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects;
[0042] The determination module is used to determine the preset network after multiple updates as a classification network.
[0043] According to a fifth aspect of the embodiments of the present invention, there is provided an object detection model building device, the device comprising:
[0044] A model acquisition module, used to obtain an original object detection model, wherein the original object detection model includes an original classification subnetwork and an original regression subnetwork;
[0045] A replacement module is used to replace the original classification subnetwork with the classification network described in the first aspect embodiment to obtain a target object detection model.
[0046] According to a sixth aspect of the embodiments of the present invention, there is provided an object detection model building device, the device comprising:
[0047] A sample image acquisition module, used to obtain multiple images of the sample object, wherein the multiple images have different image distribution domains;
[0048] A detection result acquisition module, used for inputting every two images with different image distribution domains into a preset network in pairs to obtain a plurality of first object detection results and a plurality of second object detection results;
[0049] A screening module, configured to compare the plurality of first object detection results with the plurality of second object detection results to determine a first target object detection result and a second target object detection result representing the same object category;
[0050] An updating module, configured to update the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and category label of the sample object;
[0051] The determination module is used to determine the preset network after multiple updates as the object detection model.
[0052] An embodiment of the present invention also discloses an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the method for constructing a classification network as described in the embodiment of the first aspect, or the method for constructing an object detection model as described in the second aspect, or the method for constructing an object detection model as described in the third aspect is implemented.
[0053] An embodiment of the present invention also discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the classification network construction method described in the embodiment of the first aspect of the present invention, or the object detection model construction method described in the second aspect, or the object detection model construction method described in the third aspect.
[0054] The embodiments of the present invention include the following advantages:
[0055] In an embodiment of the present invention, feature extraction can be performed on multiple images of sample objects respectively to obtain corresponding multiple feature images, wherein the multiple images have different image distribution domains; the feature image pairs corresponding to each two images with different image distribution domains are input into a preset network to obtain multiple first classification results and multiple second classification results; the preset network is updated according to the distance between the multiple first classification results and the multiple second classification results and the category label of the sample object; and the preset network after multiple updates is determined as a classification network.
[0056] In this embodiment, after the feature image pairs corresponding to each two images with different image distribution domains are input into the preset network, the distance between the multiple first classification results and the multiple second classification results can characterize the recognition gap of the preset network for images in different shooting states. When the parameters of the preset network are updated according to the category labels of the sample objects, the distance between the multiple first classification results and the multiple second classification results, the gap in the classification of the images of objects taken in different states by the preset network can be minimized as much as possible, so that the classification results of the sample objects taken in different states by the preset network can be consistent. Furthermore, the classification network obtained by subsequent training can adapt to the object detection of images taken in different shooting states, thereby improving the accuracy of object detection when the state of the shot object changes. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0058] Figure 1It is a schematic diagram of a technical concept proposed in an embodiment of the present invention;
[0059] Figure 2 is a flowchart of the steps of the classification network construction method in an embodiment of the present invention;
[0060] Figure 3 is a flow chart of the steps of updating a preset network in the implementation of the present invention;
[0061] Figure 4 is a flowchart of the steps of constructing an object detection model in the implementation of the present invention;
[0062] Figure 5 is a model structure diagram of an object detection model in the implementation of the present invention;
[0063] Figure 6 is a flow chart of the steps of updating a preset network in the implementation of the present invention;
[0064] Figure 7 It is a structural block diagram of a classification network construction device in the implementation of the present invention;
[0065] Figure 8 It is a structural block diagram of an object detection model building device in the implementation of the present invention;
[0066] Fig. 9 It is a structural block diagram of another device for constructing an object detection model in the implementation of the present invention. DETAILED DESCRIPTION
[0067] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0068] In order to solve the problems in the related art, the inventors proposed the following technical concept: during the model training process, an additional supervision is added to the classification results of images of the same type of objects taken under different shooting conditions to promote the two classification results to be close to each other in the high-dimensional space, thereby enhancing the model's detection ability for images taken under different shooting conditions.
[0069] In order to realize the above technical concept, refer to Figure 1 As shown, a schematic diagram of a technical concept proposed in an embodiment of the present application is shown.
[0070] like Figure 1As shown, during the model training process, two sample images taken for the same area under different shooting conditions can be input into the neural network, and then the classification results corresponding to the two sample images output by the neural network are obtained, and the gap between the classification results of the two is determined, so as to update the parameters of the neural network according to the gap between the classification results and the loss corresponding to the classification results. For example, the parameters of the neural network are updated according to the gap S and the losses L1 and L2 respectively related to the classification results of the two, so as to continuously narrow the gap between the classification results of the images taken under different shooting conditions during the training process, so that the classification results of the two images classified by the neural network tend to be consistent.
[0071] Combination Figure 1 The technical concept shown in the embodiment of the present application shows an example of constructing a neural network with object classification capabilities, which is introduced in Sections 1.1, 1.2 and 1.3 respectively below.
[0072] 1.1 The process of building a classification network:
[0073] Reference Figure 2 As shown, a flowchart of the steps of the classification network construction method is shown, and the method can be applied to intelligent terminals, such as Figure 2 As shown, the following steps may be specifically included:
[0074] Step S201: extracting features from a plurality of images of a sample object respectively to obtain a plurality of corresponding feature images, wherein the plurality of images have different image distribution domains.
[0075] In this embodiment, the sample object may refer to any type of object. Specifically, multiple images obtained by photographing the sample object under different shooting conditions may be obtained, and feature extraction may be performed on the multiple images to obtain feature images corresponding to the multiple images. The different shooting conditions may refer to different shooting environment conditions, different shooting picture styles, or different shooting scenes.
[0076] Since the multiple images are taken under different shooting conditions, the multiple images obtained may have different image distribution domains, and the image distribution domain in this embodiment may refer to different background information of the images. In this way, the multiple feature images obtained after feature extraction also have different background features.
[0077] For example, for different environmental conditions, the obtained images may have different background environments, such as night and day environments, and different weather conditions, such as foggy, rainy, sunny and other environmental backgrounds. Figure 1In the example, a vehicle can be photographed in clear weather to obtain image A, and a vehicle can be photographed in foggy weather to obtain image B. The two images include the same type of sample objects. After feature extraction of image A, the background feature representation of the feature image obtained is a clear weather background, while after feature extraction of the second domain sample image B, the background feature representation of the feature image obtained is a foggy weather background.
[0078] As another example, the sample object may be photographed in different picture styles, so that images of different styles of the sample object may be obtained, such as sketch-style images, oil painting-style images, etc. In this way, the images obtained after feature extraction of images of different styles may have different image distribution domains, which may also be understood as having different background styles. Of course, the images may also be taken in different background scenes, for example, images may be taken in different street scenes.
[0079] It is understandable that the multiple images obtained by photographing the sample object under different shooting conditions may also be images photographed under different camera viewing angles, that is, different images are photographed under different shooting conditions and also under different camera viewing angles.
[0080] Step S202: inputting the feature images corresponding to each two images with different image distribution domains into a preset network in pairs to obtain a plurality of first classification results and a plurality of second classification results.
[0081] In this embodiment, after obtaining multiple feature images, every two feature images with different image distribution domains can be combined into a sample pair. In this way, multiple sample pairs can be obtained, each sample pair including a feature image under one image distribution domain and a feature image under another image distribution domain.
[0082] In one scenario, the two feature images included in the sample pair may be images taken from the same camera perspective, which can reduce the difficulty of subsequent comparison of the classification results of the two. Of course, in some other scenarios, the two feature images included in the sample image pair may also be images taken from different camera perspectives.
[0083] Among them, the preset network can be a convolutional neural network. Specifically, the structure of the preset network can be set according to actual needs. The preset network can be used to identify the category to which the object in the image belongs.
[0084] In this embodiment, the two feature images in each sample pair can be input into the preset network in pairs, so as to obtain a first classification result output by the preset network for classifying the feature images in one of the image distribution domains, and a second classification result for classifying the feature images in another of the image distribution domains. That is, the first classification result represents the result of the preset network classifying objects in images taken under one shooting state, and the second classification result represents the result of the preset network classifying objects in images taken under another different shooting state.
[0085] like Figure 1 As shown, the feature image of the second domain sample image B and the feature image of the second domain sample image A can be input into the neural network for classification and recognition. In this way, the neural network classifies objects photographed under different weather conditions.
[0086] Step S203: updating the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects.
[0087] In practice, since a sample pair input to the preset network is the feature images corresponding to two images in different image distribution domains, the feature images in different image distribution domains have different background features. Therefore, the accuracy of the preset network in classifying objects for the two feature images in a sample pair may be different.
[0088] For example, Figure 1 As shown in the figure, the second domain sample image B is a car photographed under foggy weather, while the first domain sample image A is a car photographed under sunny weather. Therefore, the clarity of the feature image corresponding to the second domain sample image B and the feature image corresponding to the first domain sample image A are different. In this way, the accuracy of the object classification of the two feature images by the preset network may be different, resulting in the possibility that similar objects in different image distribution domains will not be clustered together in the feature map output by the last layer.
[0089] Therefore, in the embodiment of the present application, in order to allow similar objects in different image distribution domains to be clustered together in the feature map output by the last layer, the gap between similar objects in different external environments can be continuously reduced during the training process.
[0090] Specifically, the distances between the plurality of first classification results and the plurality of second classification results may be determined, and then when the parameters of the preset network are updated according to the first classification results, the second classification results and the category labels, the parameters of the preset network are updated in combination with the distances between the first classification results and the second classification results. The parameters of the preset network may refer to model parameters of the preset network.
[0091] It can be understood that the category label refers to the actual category to which the sample object belongs.
[0092] Step S204: determining the preset network after multiple updates as a classified network.
[0093] In this embodiment, the training can be ended when the preset number of updates is reached or the preset network reaches convergence, so that the preset network at the end of the training is used as the classification network.
[0094] In one example, referring to Figure 3 As shown, a flowchart of the steps for updating the preset network is shown, such as Figure 3 As shown, the following steps may be specifically included:
[0095] Step S2041: Determine a target loss value according to the distances between the multiple first classification results and the multiple second classification results.
[0096] In this embodiment, when the preset network is updated, there are two kinds of losses, one is the gap between the classification results, and the other is the loss determined according to the classification results and the classification labels. Among them, the gap between the classification results can be determined as the target loss value.
[0097] In this way, during each training, the distance between the first classification result and the second classification result output by the preset network of the training can be determined, and the difference can be determined as the target loss value. Therefore, the target loss value can represent the difference in the classification results of object classification for feature images of different image distribution domains. The smaller the difference, the more it represents that the preset network can overcome the influence of different image distribution domains and accurately classify objects.
[0098] In one example, when determining the target loss value, the target loss value can be determined based on the gap between the result centers of the classification results. Specifically, the multiple first classification results can be processed first to determine the first result center, and the multiple second classification results can be processed to determine the second result center; then, the target loss value can be determined based on the gap between the first result center and the second result center.
[0099] When determining the target loss value based on the gap between the first result center and the second result center, it can be determined by the following formula (1):
[0100]
[0101] Among them, Loss cosine is the target loss value, is the vector representation of the center of the first classification result, is the vector representation of the center of the second classification result.
[0102] In this embodiment, the first classification result is processed by operating the feature vector of the first classification result to obtain the mean of the feature vector, which is the center of the first result, that is, the center of the first result can be represented by a vector. Similarly, the center of the second result can be obtained. After that, the target loss value can be obtained according to formula (1).
[0103] Step S2042: Determine a first classification loss value according to the plurality of first classification results and the category label of the sample object, and determine a second classification loss value according to the plurality of second classification results and the category label of the sample object.
[0104] In this embodiment, during each training, a first classification loss value for classifying objects in a feature image of one image distribution domain can be determined based on the first classification result and category label outputted by the training. A second classification loss value for classifying objects in a feature image of another image distribution domain can also be determined based on the second classification result and category label outputted by the training.
[0105] Among them, the first classification loss value and the second classification loss value can both represent the gap between the classification result of the sample object and the true category.
[0106] Step S2043: updating the parameters of the preset network according to the target loss value, the first classification loss value and the second classification loss value.
[0107] In this embodiment, the overall loss of the preset network can be determined according to the target loss value, the first classification loss value, the second classification loss value and their corresponding weights, and then the parameters of the preset network can be updated according to the overall loss.
[0108] The technical solution of the embodiment of the present application can obtain a trained classification network, so that the classification network can be used to classify objects, and can also be used as a subnetwork in other object detection networks to identify the category of objects.
[0109] 1.2 The construction process of building an object detection model.
[0110] There are two ways to construct an object detection model. One is to directly replace the classification subnetwork in the original object detection model with the classification network trained in this application as described in Section 1.2.1. The other is to train a preset network including a classification branch and a positioning branch according to the training process of training the classification network as described in Section 1.2.2, thereby training an object detection model.
[0111] 1.2.1 Method 1 for building object detection model
[0112] First, an original object detection model is obtained, where the original object detection model includes an original classification subnetwork and an original regression subnetwork.
[0113] Next, the original classification subnetwork is replaced with the classification network trained through the process in Section 1.1 to obtain a target object detection model.
[0114] In this embodiment, the regression sub-network can generate a location box of the sample object, and the classification sub-network can be used to classify the object in the location box.
[0115] Of course, in one example, the original object detection model may also include a feature extraction module, wherein the original classification subnetwork and the original regression subnetwork are respectively connected to the output end of the feature extraction module, and the feature extraction module is used to extract features from the image input to the original object detection model, and simultaneously input the extracted feature image to the original classification subnetwork and the original regression subnetwork. After replacing the original classification subnetwork with the classification network trained by the process in Section 1.1, the extracted feature image can be input to the classification network and the original regression subnetwork.
[0116] Among them, after replacing the original classification subnetwork with the classification network trained through the process in Section 1.1, the target object detection model can be trained using multiple sample images of the sample object taken under different shooting conditions to obtain the final target object detection model, which can further improve the object classification accuracy of the target object detection model.
[0117] It can be understood that when the classification network is replaced with the original object detection model as the classification subnetwork, the change of the original regression subnetwork is avoided. Therefore, the classification network of the present application can be replaced with a network model with object classification function, thereby improving the applicability of the classification network.
[0118] 1.2.2 Method 2 for building object detection model
[0119] Reference Figure 4 and Figure 5 As shown, Figure 4 Outputs a flowchart of the steps to build an object detection model. Figure 5 FIG. 2 shows a schematic diagram of the principle of training an object detection model in the second method. Figure 4 As shown, the following steps may be specifically included:
[0120] Step S401: obtaining a plurality of images of a sample object, wherein the plurality of images have different image distribution domains.
[0121] In this embodiment, the process of obtaining multiple images of the sample object is as described in the above step S201, which will not be described in detail here.
[0122] Each image may include multiple sample objects. Figure 5 As shown, Figure 1 Each image input to the neural network on the left side includes multiple vehicles, that is, image A and image B each include 5 sample objects, that is, 5 vehicles, and each vehicle is a sample object.
[0123] Step S402: Inputting every two images with different image distribution domains into a preset network in pairs to obtain a plurality of first object detection results and a plurality of second object detection results.
[0124] like Figure 5 As shown, the preset network may include a backbone network, a classification branch and a positioning branch. The backbone network is used to extract features from each input image, and the feature images obtained after feature extraction are input into the classification branch and the positioning branch respectively.
[0125] Among them, the classification branch is used to process the input feature image to obtain the category of the sample object, and the positioning branch is used to process the input feature image to obtain the location of the sample object. In this embodiment, for the two images input in pairs, the category and position of each sample object in each image can be obtained, wherein the category and position of a sample object are the object detection results. In this way, since an image includes multiple sample objects, for each image, multiple corresponding object detection results can be obtained. Among them, multiple first object detection results correspond to one of the images input into the preset network in pairs, and multiple second object detection results correspond to another image input into the preset network in pairs.
[0126] Step S403: Compare the multiple first object detection results with the multiple second object detection results to determine a first target object detection result and a second target object detection result that represent the same object category.
[0127] In this embodiment, for an image input to the preset network, the categories and positions of multiple sample objects in the image can be detected, thereby obtaining multiple first object detection results, each of which corresponds to the category and position of a sample object. Similarly, for another image input to the preset network, multiple second object detection results can be obtained, each of which corresponds to the category and position of a sample object.
[0128] Among them, the first target object detection result and the second target object detection result of the same object category can be determined from the multiple first object detection results and the multiple second object detection results. That is, according to the object category represented by the first object detection result and the object category represented by the second object detection result, the first object detection result and the second object detection result of the same object category are screened out to find the same type of objects in different image distribution domains.
[0129] like Figure 5 As shown, since both image A and image B include 5 sample objects, 5 object detection results can be obtained for each image. One object detection result includes the position and category of a vehicle. Since the 5 sample objects belong to the same category, namely, the vehicle category, the object detection result belonging to the vehicle in the first object detection result can be used as the first target object detection result, and the object detection result belonging to the vehicle in the second object detection result can also be used as the second target object detection result.
[0130] Step S404: updating the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and category label of the sample object.
[0131] Step S405: determining the preset network after multiple updates as the object detection model.
[0132] Among them, after screening out the first target object detection result and the second target object detection result of the same object category, since it is necessary to cluster similar objects in different image distribution domains together, the gap between similar objects from different image distribution domains can be continuously reduced during the training process, that is, the distance between the first target object detection result and the second target object detection result of the same object category can be continuously reduced.
[0133] For example, Figure 1 As shown, the first target object detection result and the second target object detection result belonging to the same vehicle need to be brought close together so that the features belonging to the vehicle can be clustered together in the high-dimensional feature map.
[0134] In a specific implementation, the distance between the first object target detection result and the second target object detection result can be used as a loss for updating the preset network, and the parameters of the preset network can be updated. Specifically, since the first object detection result includes a position detection result and a category detection result, and the second object detection result also includes a position detection result and a category detection result, the loss of object detection can be determined based on the first object detection result, the second object detection result, the position detection result, and the category detection result. In this way, when updating the preset network, the preset network can be updated multiple times based on the loss of object detection and the distance between the first object target detection result and the second target object detection result.
[0135] In one example, referring to Figure 6 As shown, a flowchart of the steps for updating the preset network is shown, such as Figure 6 As shown, the following steps may be specifically included:
[0136] Step S601: determining a target loss value according to a distance between the first target object detection result and the second target object detection result.
[0137] In this embodiment, the process of determining the target loss value can refer to the process described in Section 1.1 above, and will not be repeated here.
[0138] Step S602: Determine a first object detection loss value based on the multiple first object detection results and the position labels and category labels of the sample objects, and determine a second object detection loss value based on the multiple second object detection results and the position labels and category labels of the sample objects.
[0139] like Figure 5 As shown, in this embodiment, since the first object detection result and the second object detection result both include position detection results and category detection results, during each training, the first position loss value L3 can be determined based on the position detection results and position labels in the multiple first object detection results, and the first category loss value L1 can be determined based on the category detection results and category labels in the multiple first object detection results, thereby determining the first object detection loss value based on the first position loss value L3 and the first category loss value L1.
[0140] Similarly, the second position loss value L4 can be determined based on the position detection results and position labels in the multiple second object detection results, and the second category loss value L2 can be determined based on the category detection results and category labels in the multiple second object detection results, thereby determining the second object detection loss value based on the second position loss value L4 and the second category loss value L2.
[0141] Step S603: updating the parameters of the preset network according to the target loss value, the first object detection loss value and the second object detection loss value.
[0142] In one example, the overall loss of the preset network can be determined according to the target loss value, the first object detection loss value, the second object detection loss value, and their corresponding weights, so that the preset network is updated according to the overall loss.
[0143] Among them, Figure 5 As shown, in an application scenario, since the preset network has a classification branch and a positioning branch, when updating, the parameters of the classification branch can be updated according to the first category loss value L1, the second category loss value L2 and the target loss value S; then, the parameters of the positioning branch can be updated according to the first position loss value L3 and the second position loss value L4.
[0144] In one example, after obtaining the object detection model, the object detection model can be used to perform object detection. Specifically, an image to be detected can be obtained, and the image to be detected has any image distribution domain in the different image distribution domains; and the image to be detected is input into the object detection model to obtain the position and category of the object contained in the image to be processed.
[0145] In this example, after the object detection model is obtained, the object detection model can perform object detection on the input image to be detected. The image to be detected can be an image taken in any shooting state, for example, an image taken in any external environment, and the image to be detected can contain an object to be identified.
[0146] Among them, the object detection model can be used to identify the position and category of the object contained in the image to be processed to obtain the detection results of the position and category of the object.
[0147] Among them, since sample images taken under different shooting conditions are used as training samples when training the object detection model, multiple images in the training samples have different image distribution domains, and during the training process, the parameters of the preset network are updated according to the gap between the detection results of the same objects under different image distribution domains. Therefore, as the training progresses, the gap between the detection results of the same objects under different image distribution domains becomes smaller and smaller. Thus, at the end of the training, the detection results of the same objects under different image distribution domains can be clustered together, so that the object detection model can adapt to the detection of images taken under different shooting conditions, improve the generalization ability of the model, and the accuracy of object detection when the shooting state changes.
[0148] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0149] Reference Figure 7 As shown, a structural block diagram of a classification network construction device according to an embodiment of the present invention is shown. Figure 7 As shown, the device may specifically include the following modules:
[0150] A feature extraction module 701 is used to extract features from multiple images of a sample object respectively to obtain corresponding multiple feature images, wherein the multiple images have different image distribution domains;
[0151] The classification result obtaining module 702 is used to input the feature images corresponding to each two images with different image distribution domains into a preset network in pairs to obtain a plurality of first classification results and a plurality of second classification results;
[0152] An updating module 703, configured to update the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects;
[0153] The determination module 704 is used to determine the preset network after multiple updates as a classified network.
[0154] The updating module 703 may specifically include the following units:
[0155] a first loss value determining unit, configured to determine a target loss value according to distances between the plurality of first classification results and the plurality of second classification results;
[0156] a second loss value determining unit, configured to determine a first classification loss value according to the plurality of first classification results and the category label of the sample object, and to determine a second classification loss value according to the plurality of second classification results and the category label of the sample object;
[0157] The third loss value determining unit is used to update the parameters of the preset network according to the target loss value, the first classification loss value and the second classification loss value.
[0158] Optionally, the device may further include the following modules:
[0159] A processing module, configured to process the plurality of first classification results to determine a first result center, and to process the plurality of second classification results to determine a second result center;
[0160] The first loss value determining unit is specifically configured to determine a target loss value according to a gap between the first result center and the second result center.
[0161] Optionally, the first loss value determining unit is specifically configured to determine the target loss value according to the following formula:
[0162]
[0163] Among them, Loss cosine is the target loss value, is the vector representation of the center of the first classification result, and is the vector representation of the center of the second classification result.
[0164] Reference Figure 8 As shown, a structural block diagram of an object detection model building device according to an embodiment of the present application is shown. Figure 8 As shown, the device may specifically include the following modules:
[0165] A model acquisition module 801 is used to obtain an original object detection model, wherein the original object detection model includes an original classification subnetwork and an original regression subnetwork;
[0166] The replacement module 802 is used to replace the original classification subnetwork with the classification network described in the embodiment of Section 1.1 to obtain a target object detection model.
[0167] Reference Fig. 9 As shown, a structural block diagram of a device for constructing an object detection model according to an embodiment of the present application is shown. Fig. 9 As shown, the device may specifically include the following modules:
[0168] A sample image acquisition module 901, used to obtain multiple images of a sample object, wherein the multiple images have different image distribution domains;
[0169] A detection result acquisition module 902, configured to input two images having different image distribution domains into a preset network in pairs, to obtain a plurality of first object detection results and a plurality of second object detection results;
[0170] A screening module 903, configured to compare the plurality of first object detection results with the plurality of second object detection results to determine a first target object detection result and a second target object detection result representing the same object category;
[0171] An updating module 904, configured to update the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and category label of the sample object;
[0172] The determination module 905 is used to determine the preset network after multiple updates as the object detection model.
[0173] Optionally, the updating module 904 may specifically include the following units:
[0174] a first loss determining unit, configured to determine a target loss value according to a distance between the first target object detection result and the second target object detection result;
[0175] a second loss determination unit, configured to determine a first object detection loss value according to the plurality of first object detection results and the position labels and the category labels of the sample objects, and to determine a second object detection loss value according to the plurality of second object detection results and the position labels and the category labels of the sample objects;
[0176] An updating unit is used to update the parameters of the preset network according to the target loss value, the first object detection loss value and the second object detection loss value.
[0177] Optionally, the device may further include the following modules:
[0178] An image acquisition module, used for acquiring an image to be detected, wherein the image to be detected has any image distribution domain among the different image distribution domains;
[0179] The detection module is used to input the image to be detected into the object detection model to obtain the position and category of the object contained in the image to be processed.
[0180] It should be noted that the device embodiment is similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the method embodiment.
[0181] An embodiment of the present invention also provides an electronic device, which may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the classification network construction method or the object detection model construction method.
[0182] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program that enables a processor to execute the classification network construction method or the object detection model construction method as described in the embodiment of the present invention.
[0183] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0184] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0185] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0186] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0188] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0189] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0190] The above is a detailed introduction to a classification network construction method, an object detection model construction method, an apparatus, a device and a storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A classification network construction method, characterized in that: The method comprises: Extracting features from multiple images of the sample object respectively to obtain corresponding multiple feature images, wherein the multiple images have different image distribution domains; Inputting feature images corresponding to each two images with different image distribution domains into a preset network in pairs to obtain a plurality of first classification results and a plurality of second classification results; updating the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects; Determine the preset network after multiple updates as a classification network; The updating of the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects comprises: Determining a target loss value according to the distances between the plurality of first classification results and the plurality of second classification results; Determining a first classification loss value according to the plurality of first classification results and the category label of the sample object, and determining a second classification loss value according to the plurality of second classification results and the category label of the sample object; The parameters of the preset network are updated according to the target loss value, the first classification loss value and the second classification loss value.
2. The method according to claim 1, characterized in that The method further comprises: Processing the plurality of first classification results to determine a first result center, and processing the plurality of second classification results to determine a second result center; Determining a target loss value according to the distances between the plurality of first classification results and the plurality of second classification results includes: A target loss value is determined according to the distance between the first result center and the second result center.
3. The method according to claim 2, characterized in that Determining a target loss value according to a distance between the first result center and the second result center includes: The target loss value is determined according to the following formula: Loss cosine = * / ( * ) Among them, Loss cosine is the target loss value, is the vector representation of the center of the first result, is the vector representation of the center of the second result.
4. A method for constructing an object detection model, characterized in that: The method comprises: Obtaining an original object detection model, wherein the original object detection model includes an original classification subnetwork and an original regression subnetwork; The original classification subnetwork is replaced with the classification network described in any one of claims 1-3 to obtain a target object detection model.
5. A method for constructing an object detection model, characterized in that: The method comprises: Acquire a plurality of images of the sample object, wherein the plurality of images have different image distribution domains; Inputting every two images with different image distribution domains into a preset network to obtain a plurality of first object detection results and a plurality of second object detection results; Comparing the plurality of first object detection results with the plurality of second object detection results to determine a first target object detection result and a second target object detection result representing the same object category; Update the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and the category label of the sample object; Determine the preset network after multiple updates as the object detection model; The updating of the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and the category label of the sample object, includes: determining a target loss value according to a distance between the first target object detection result and the second target object detection result; Determine a first object detection loss value according to the plurality of first object detection results and the position labels and the category labels of the sample objects, and determine a second object detection loss value according to the plurality of second object detection results and the position labels and the category labels of the sample objects; The parameters of the preset network are updated according to the target loss value, the first object detection loss value, and the second object detection loss value.
6. The method according to claim 5, characterized in that After obtaining the object detection model, the method further includes: Obtaining an image to be detected, wherein the image to be detected has any image distribution domain among the different image distribution domains; The image to be detected is input into the object detection model to obtain the position and category of the object contained in the image to be detected.
7. A classification network construction device, characterized in that: The device comprises: A feature extraction module, used to extract features from multiple images of the sample object respectively to obtain corresponding multiple feature images, wherein the multiple images have different image distribution domains; A classification result acquisition module, used for inputting the feature images corresponding to each two images with different image distribution domains into a preset network in pairs to obtain a plurality of first classification results and a plurality of second classification results; An updating module, configured to update the parameters of the preset network according to the distances between the plurality of first classification results and the plurality of second classification results and the category labels of the sample objects; A determination module, used to determine the preset network after multiple updates as a classification network; The update module is specifically used for: Determining a target loss value according to the distances between the plurality of first classification results and the plurality of second classification results; Determining a first classification loss value according to the plurality of first classification results and the category label of the sample object, and determining a second classification loss value according to the plurality of second classification results and the category label of the sample object; The parameters of the preset network are updated according to the target loss value, the first classification loss value and the second classification loss value.
8. An object detection model building device, characterized in that: The device comprises: A model acquisition module, used to obtain an original object detection model, wherein the original object detection model includes an original classification subnetwork and an original regression subnetwork; A replacement module is used to replace the original classification subnetwork with the classification network described in any one of claims 1-3 to obtain a target object detection model.
9. An object detection model building device, characterized in that: The device comprises: A sample image acquisition module, used to obtain multiple images of the sample object, wherein the multiple images have different image distribution domains; A detection result acquisition module, used for inputting every two images with different image distribution domains into a preset network in pairs to obtain a plurality of first object detection results and a plurality of second object detection results; A screening module, configured to compare the plurality of first object detection results with the plurality of second object detection results to determine a first target object detection result and a second target object detection result representing the same object category; An updating module, configured to update the parameters of the preset network according to the distance between the first target object detection result and the second target object detection result, and the position label and the category label of the sample object; A determination module, used to determine the preset network after multiple updates as an object detection model; The update module is specifically used for: determining a target loss value according to a distance between the first target object detection result and the second target object detection result; Determine a first object detection loss value according to the plurality of first object detection results and the position labels and the category labels of the sample objects, and determine a second object detection loss value according to the plurality of second object detection results and the position labels and the category labels of the sample objects; The parameters of the preset network are updated according to the target loss value, the first object detection loss value, and the second object detection loss value.
10. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When executed by the processor, the method for constructing a classification network as described in any one of claims 1 to 3, or the method for constructing an object detection model as described in claim 4, or the method for constructing an object detection model as described in any one of claims 5 to 6 is implemented.
11. A computer-readable storage medium, characterized in that: The computer program stored therein enables the processor to execute the classification network construction method as described in any one of claims 1-3, or the object detection model construction method as described in claim 4, or the object detection model construction method as described in any one of claims 5-6.
Citation Information
Patent Citations
Face image gender judging method and device, computer equipment and storage medium
CN108446688A
