A target detection model training method, a target detection method, and a device

Abstract

The present disclosure relates to a target detection model training method, a target detection method and device, and relates to the technical field of computers. The method comprises: determining initial features of each of a plurality of detection boxes included in a detected picture and at least one weight feature corresponding to the initial features of each of the detection boxes; generating target features of each of the detection boxes based on the initial features of each of the detection boxes and the at least one weight feature; and training a first target detection model based on the target features of each of the detection boxes to generate a second target detection model. In the present disclosure, the electronic device can realize the migration from the classification picture to the category on the detected picture, without a large amount of manual labeling on the training samples of the target detection model, can accurately generate the target features capable of representing the real data distribution of the category of each of the detection boxes, and can improve the training efficiency of the target detection model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a target detection model training method, a target detection method, and a target detection device. Background Art

[0002] Currently, in fields such as target detection, electronic devices can input a target detection picture into a trained neural network model, and then the neural network model can predict the multiple targets (or objects) included in the target detection picture and the respective categories of the multiple targets.

[0003] However, during the training process of the above-mentioned neural network model, a large number of high-quality target detection training samples may be required. Specifically, a large amount of manpower and material resources are required to label the categories of each target included in each target detection training sample, etc. This may affect the training efficiency of the neural network model and thus reduce the prediction efficiency of the neural network model. Summary of the Invention

[0004] The present disclosure provides a target detection model training method, a target detection method and a device, which solves the technical problem in related technologies that a large amount of manpower and material resources are required to label the categories of each target included in each target detection training sample, which may affect the training efficiency of the neural network model and thus reduce the prediction efficiency of the neural network model.

[0005] The technical solutions of the embodiments of the present disclosure are as follows:

[0006] According to the first aspect of the embodiment of the present disclosure, a method for training a target detection model is provided. The method may include: determining the initial features of each detection frame in a plurality of detection frames included in a detected image and at least one weight feature corresponding to the initial features of each detection frame, wherein the similarity between each weight feature in the at least one weight feature corresponding to the initial features of a detection frame and the initial features of the detection frame is greater than or equal to a similarity threshold, and the at least one weight feature corresponding to the initial features of each detection frame is a feature of a weight included in a first classifier, and a weight corresponds to a category in the first classifier; based on the initial features of each detection frame in the plurality of detection frames and the at least one weight feature corresponding to the initial features of each detection frame, generating a target feature of each detection frame; based on the target features of each detection frame, training the first target detection model to generate a second target detection model.

[0007] Optionally, the above-mentioned first target detection model includes a first sub-network model and a second sub-network model, and the above-mentioned determination of the initial features of each detection frame in the multiple detection frames included in the detected picture specifically includes: obtaining the detected picture; inputting the detected picture into the first sub-network model, performing feature recognition, and obtaining the initial features of the detected picture; inputting the initial features of the detected picture into the second sub-network model to obtain the position of each detection frame in the multiple detection frames and the size of each detection frame; based on the position of each detection frame and the size of each detection frame, extracting the initial features of each detection frame from the initial features of the detected picture.

[0008] Optionally, the above-mentioned first classifier is a classifier included in a first image classification model, which includes a third sub-network model, and the parameters in the third sub-network model are the same as the parameters in the first sub-network model. The training method of the above-mentioned target detection model also includes: obtaining multiple classified images; inputting the multiple classified images into the third sub-network model to obtain the initial features of each classified image in the multiple classified images; inputting the initial features of each classified image into the first classifier to obtain the predicted classification results of each classified image in the first classifier.

[0009] Optionally, the training method of the target detection model further includes: determining a first loss and a second loss, the first loss being used to characterize the degree of inconsistency between the actual classification result of each classified image in a plurality of classified images and the predicted classification result of each classified image in the first classifier, and the second loss being used to characterize the degree of spatial inconsistency of each detection frame in the plurality of detection frames, wherein the spatial inconsistency of a detection frame includes the degree of inconsistency between the actual position of the detection frame and the predicted position of the detection frame in the first target detection model, and the degree of inconsistency between the actual size of the detection frame and the predicted size of the detection frame in the first target detection model; the training of the first target detection model based on the target features of each detection frame to generate the second target detection model specifically includes: training each detection frame The target feature of a detection frame is input into the second classifier to obtain the predicted classification result of each detection frame in the second classifier, and the second classifier is a classifier included in the first target detection model; a third loss and a fourth loss are determined, the third loss is used to characterize the degree of inconsistency between the true classification result of each detection frame and the predicted classification result of each detection frame in the second classifier, and the fourth loss is used to characterize the degree of inconsistency between the true result of whether each detection frame includes an object and the result of whether each detection frame is predicted to include an object in the second classifier; a target loss is determined based on the first loss, the second loss, the third loss and the fourth loss; based on the target loss, the parameters in the first target detection model are updated to generate the second target detection model.

[0010] Optionally, the training method of the above-mentioned target detection model also includes: determining the target feature of the first detection frame as a preset mean, and the first detection frame is one of the multiple detection frames; generating multiple extended features of the first detection frame based on the preset mean and the preset standard deviation; and training the first target detection model based on the multiple extended features of each detection frame in the multiple detection frames.

[0011] According to a second aspect of an embodiment of the present disclosure, a method for object detection is provided. The method may include: obtaining an image to be detected; inputting the image to be detected into a second object detection model to obtain a category of at least one detection box included in the image to be detected, wherein the second object detection model is trained using any of the optional object detection model training methods described in the first aspect.

[0012] Optionally, the above-mentioned second target detection model includes a first sub-network model, a second sub-network model and a second classifier. The above-mentioned inputting the picture to be detected into the second target detection model to obtain the respective categories of at least one detection box included in the picture to be detected specifically includes: inputting the picture to be detected into the first sub-network model, performing feature recognition, and obtaining the initial features of the picture to be detected; inputting the initial features of the picture to be detected into the second sub-network model to obtain the respective positions of the at least one detection box and the respective sizes of the at least one detection box; based on the respective positions of the at least one detection box and the respective sizes of the at least one detection box, extracting the respective initial features of the at least one detection box from the initial features of the picture to be detected; inputting the respective initial features of the at least one detection box into the second classifier to obtain the respective categories of the at least one detection box.

[0013] According to a third aspect of an embodiment of the present disclosure, a training device for a target detection model is provided. The device may include: a determination module and a processing module; the determination module is configured to determine the initial features of each detection frame in a plurality of detection frames included in a detected image and at least one weight feature corresponding to the initial features of each detection frame, wherein the similarity between each weight feature in the at least one weight feature corresponding to the initial features of a detection frame and the initial features of the detection frame is greater than or equal to a similarity threshold, and the at least one weight feature corresponding to the initial features of each detection frame is a feature of a weight included in a first classifier, and a weight corresponds to a category in the first classifier; the processing module is configured to generate a target feature of each detection frame based on the initial features of each detection frame and the at least one weight feature corresponding to the initial features of each detection frame; the processing module is further configured to train the first target detection model based on the target features of each detection frame to generate a second target detection model.

[0014] Optionally, the above-mentioned first target detection model includes a first sub-network model and a second sub-network model, and the training device of the above-mentioned target detection model also includes an acquisition module; the acquisition module is configured to acquire the detected image; the processing module is specifically configured to input the detected image into the first sub-network model, perform feature recognition, and obtain the initial features of the detected image; the processing module is specifically configured to input the initial features of the detected image into the second sub-network model to obtain the position of each detection frame in the multiple detection frames and the size of each detection frame; the processing module is specifically configured to extract the initial features of each detection frame from the initial features of the detected image based on the position of each detection frame and the size of each detection frame.

[0015] Optionally, the above-mentioned first classifier is a classifier included in a first image classification model, which includes a third sub-network model, and the parameters in the third sub-network model are the same as the parameters in the first sub-network model; the acquisition module is also configured to acquire multiple classified images; the processing module is also configured to input the multiple classified images into the third sub-network model to obtain the initial features of each classified image in the multiple classified images; the processing module is also configured to input the initial features of each classified image into the first classifier to obtain the predicted classification results of each classified image in the first classifier.

[0016] Optionally, the determination module is further configured to determine a first loss and a second loss, the first loss being used to characterize the degree of inconsistency between the true classification result of each classified image in the multiple classified images and the predicted classification result of each classified image in the first classifier, and the second loss being used to characterize the degree of spatial inconsistency of each detection frame in the multiple detection frames, wherein the spatial inconsistency degree of a detection frame includes the degree of inconsistency between the true position of the detection frame and the predicted position of the detection frame in the first target detection model, and the degree of inconsistency between the true size of the detection frame and the predicted size of the detection frame in the first target detection model; the processing module is specifically configured to input the target feature of each detection frame into the second classifier, and obtain the predicted size of each detection frame in the second classifier. classification result, the second classifier is the classifier included in the first target detection model; the determination module is specifically configured to determine a third loss and a fourth loss, the third loss is used to characterize the degree of inconsistency between the true classification result of each detection frame and the predicted classification result of each detection frame in the second classifier, and the fourth loss is used to characterize the degree of inconsistency between the true result of whether each detection frame includes an object and the result of whether each detection frame is predicted to include an object in the second classifier; the determination module is further specifically configured to determine a target loss based on the first loss, the second loss, the third loss and the fourth loss; the processing module is further specifically configured to update the parameters in the first target detection model based on the target loss to generate the second target detection model.

[0017] Optionally, the determination module is further configured to determine the target feature of the first detection frame as a preset mean, and the first detection frame is one of the multiple detection frames; the processing module is further configured to generate multiple extended features of the first detection frame based on the preset mean and the preset standard deviation; the processing module is further configured to train the first target detection model based on the multiple extended features of each detection frame in the multiple detection frames.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a target detection device is provided. The device may include: an acquisition module and a processing module; the acquisition module is configured to acquire an image to be detected; the processing module is configured to input the image to be detected into a second target detection model to obtain the category of at least one detection box included in the image to be detected, wherein the second target detection model is trained based on any of the optional target detection model training methods described in the first aspect.

[0019] Optionally, the second target detection model includes a first sub-network model, a second sub-network model and a second classifier; the processing module is specifically configured to input the image to be detected into the first sub-network model, perform feature recognition, and obtain the initial features of the image to be detected; the processing module is also specifically configured to input the initial features of the image to be detected into the second sub-network model to obtain the respective positions of the at least one detection frame and the respective sizes of the at least one detection frame; the processing module is also specifically configured to extract the respective initial features of the at least one detection frame from the initial features of the image to be detected based on the respective positions of the at least one detection frame and the respective sizes of the at least one detection frame; the processing module is also specifically configured to input the respective initial features of the at least one detection frame into the second classifier to obtain the respective categories of the at least one detection frame.

[0020] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, which may include: a processor and a memory configured to store processor-executable instructions; wherein the processor is configured to execute the instructions to implement any one of the optional target detection model training methods in the above-mentioned first aspect, or to implement any one of the optional target detection methods in the above-mentioned second aspect.

[0021] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which instructions are stored. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the optional target detection model training methods in the above-mentioned first aspect, or execute any one of the optional target detection methods in the above-mentioned second aspect.

[0022] According to the seventh aspect of an embodiment of the present disclosure, a computer program product is provided, which includes computer instructions. When the computer instructions are executed on a processor of an electronic device, the electronic device executes any optional target detection model training method as in the first aspect, or executes any optional target detection method as in the second aspect.

[0023] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0024] Based on any of the above aspects, in the present disclosure, the electronic device can determine the initial features of each detection frame in the multiple detection frames included in the detected image and at least one weight feature corresponding to each detection frame, and generate the target features of each detection frame based on the initial features of each detection frame and the at least one weight feature corresponding to each detection frame. Since the at least one weight feature corresponding to each detection frame is a feature of the weight included in the first classifier, a weight corresponds to a category in the first classifier, and the similarity between the at least one weight feature corresponding to a detection frame and the detection frame is greater than or equal to the similarity threshold, the electronic device can determine at least one category (i.e., the category corresponding to each of the at least one weight feature corresponding to the initial features of the detection frame) that is relatively similar to the category of the detection frame (specifically, the category of the object included in the detection frame) from the multiple categories included in the first classifier. The electronic device can then combine the features corresponding to the category of the detection frame (i.e., the initial features of the detection frame) with the features corresponding to the at least one category (i.e., at least one weight feature corresponding to the initial features of the detection frame), thereby realizing the migration of categories from the classified image to the detection image. This eliminates the need for a large amount of manual annotation of the training samples of the target detection model, and can accurately generate target features that can characterize the real data distribution of the category of each detection frame (specifically, the category of the object included in each detection frame). The electronic device can then train the first target detection model based on the target features of each detection frame to generate a second target detection model, thereby improving the training efficiency of the target detection model. Furthermore, the electronic device can quickly and accurately predict the categories of multiple objects included in each image to be detected based on the second target detection model, thereby improving the prediction efficiency of the target detection model.

[0025] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0027] Figure 1 A schematic diagram of a process for training a target detection model provided by an embodiment of the present disclosure is shown;

[0028] Figure 2 A schematic diagram illustrating a flow chart of another method for training a target detection model provided by an embodiment of the present disclosure is shown;

[0029] Figure 3A schematic diagram illustrating a flow chart of another method for training a target detection model provided by an embodiment of the present disclosure is shown;

[0030] Figure 4 A schematic diagram illustrating a flow chart of another method for training a target detection model provided by an embodiment of the present disclosure is shown;

[0031] Figure 5 A schematic diagram illustrating a flow chart of another method for training a target detection model provided by an embodiment of the present disclosure is shown;

[0032] Figure 6 A schematic diagram of a process of a target detection method provided by an embodiment of the present disclosure is shown;

[0033] Figure 7 A schematic diagram showing a flow chart of another target detection method provided by an embodiment of the present disclosure is shown;

[0034] Figure 8 A schematic diagram of the structure of a training device for an object detection model provided by an embodiment of the present disclosure is shown;

[0035] Figure 9 A schematic diagram of the structure of another target detection model training device provided by an embodiment of the present disclosure is shown;

[0036] Figure 10 A schematic structural diagram of a target detection device provided by an embodiment of the present disclosure is shown;

[0037] Figure 11 A schematic structural diagram of another target detection device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0038] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0039] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0040] It will also be understood that the term “comprising” indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements and / or components.

[0041] It should be noted that the user information (including but not limited to user device information, user personal information, user behavior information, etc.) and data (including but not limited to detected images, classified images, and images to be detected, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0042] In related technologies, a large number of high-quality target detection training samples may be required during the training of a neural network model. Specifically, a large amount of manpower and material resources are required to label the categories of each target included in each target detection training sample, which may affect the training efficiency of the neural network model and thus reduce the prediction efficiency of the neural network model.

[0043] Based on this, the embodiment of the present disclosure provides a method for training a target detection model. The electronic device can combine the features corresponding to the category of a detection frame in the multiple detection frames included in the detected image (i.e., the initial features of the detection frame) with the features corresponding to at least one category included in the first classifier (i.e., at least one weight feature corresponding to the initial features of the detection frame). This can achieve the migration of categories from the classified image to the detected image, without the need for a large amount of manual labeling of the training samples of the target detection model. The target features that can characterize the real data distribution of the category of each detection frame (specifically, the category of the object included in each detection frame) can be accurately generated. And the electronic device can train the first target detection model based on the target features of each detection frame to generate a second target detection model, which can improve the training efficiency of the target detection model. Furthermore, the electronic device can quickly and accurately predict the categories of multiple objects included in each image to be detected based on the second target detection model, that is, it can improve the prediction efficiency of the target detection model.

[0044] The training method, target detection method and device of the target detection model provided by the embodiments of the present disclosure are applied to target detection scenarios (specifically, scenarios where the category of each object (or target) included in a certain picture to be detected needs to be determined). When the electronic device determines the initial features of each detection frame in the multiple detection frames included in the detected picture and at least one weight feature corresponding to the initial features of each detection frame, the first target detection model (which can be understood as the target detection model in the initial state) can be trained to generate a second target detection model (which can be understood as the target detection model that has been trained) according to the method provided by the embodiments of the present disclosure. Furthermore, when the electronic device obtains the picture to be detected, the picture to be detected can be input into the second target detection model to obtain the respective categories of at least one detection frame included in the picture to be detected (which can also be understood as the categories of the objects or targets included in the at least one detection frame).

[0045] The following is an exemplary description of the target detection model training method and the target detection method provided by the embodiments of the present disclosure with reference to the accompanying drawings:

[0046] Exemplarily, the electronic device that executes the target detection model training method and target detection method provided by the embodiments of the present disclosure can be a mobile phone, tablet computer, desktop, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR)\virtual reality (VR) device, etc., which can install and use content community applications. The present disclosure does not impose any special restrictions on the specific form of the electronic device. It can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device.

[0047] like Figure 1 As shown, the training method of the target detection model provided by the embodiment of the present disclosure may include S101-S103.

[0048] S101: The electronic device determines an initial feature of each detection frame in a plurality of detection frames included in a detected image and at least one weight feature corresponding to the initial feature of each detection frame.

[0049] Among them, the similarity between each weight feature of at least one weight feature corresponding to the initial feature of a detection frame and the initial feature of the detection frame is greater than or equal to the similarity threshold, the weight feature corresponding to the initial feature of each detection frame is the feature of the weight included in the first classifier, and one weight corresponds to a category in the first classifier.

[0050] It should be understood that one of the multiple detection frames described above represents an area in the detected image. The detection frame (or the area) may or may not include an object (or target). When the detection frame includes an object, the category of the object is the category of the detection frame. When the detection frame does not include an object, it indicates that the detection frame represents a background area in the detected image.

[0051] Optionally, the detection box in the embodiment of the present disclosure may be a proposal box.

[0052] It can be understood that the above-mentioned first classifier is a classifier included in the first image classification model, and the first image classification model is an image classification model in a certain initial state. The image classification model is used to classify a certain classification image, specifically to determine (or predict) the category corresponding to the classification image.

[0053] It should be noted that the detection image and the classification image in the embodiment of the present disclosure are different images and act on different network models respectively.

[0054] Specifically, the detection image is generally large, the scene represented by the detection image is relatively complex, and the detection image may include multiple objects (or targets). For each of the multiple objects, a detection frame can be configured for it, thereby making each object independent. The electronic device inputs the detection image into a target detection model to obtain the category of each detection frame. For a detection frame containing an object, the category of the detection frame ends with the category of the object contained in the detection frame.

[0055] Classification images are generally small, representing simple scenes and typically contain only one object. The electronic device inputs the classification image into an image classification model to obtain the category of the classification image, which is the category of the object contained in the classification image.

[0056] In an embodiment of the present disclosure, a certain image classification model (such as the first image classification model mentioned above) may include a classifier (i.e., a first classifier). The first classifier may be a multi-class classifier, that is, it can divide multiple classified images into at least three categories.

[0057] It should be understood that the first classifier includes multiple categories, one category corresponds to one weight, and the electronic device can obtain the features of the weight corresponding to each category in the multiple categories, and then determine the similarity between the features of the weight corresponding to each category and the initial features of a detection frame in the above-mentioned multiple detection frames. When the similarity between the initial features of the detection frame and the features of the weight corresponding to a category is greater than or equal to the similarity threshold, it means that the category of the detection frame (specifically, the category of the object included in the detection frame) is relatively similar to the category. At this time, the electronic device can determine the feature of the weight corresponding to the category as one of the at least one weight features corresponding to the initial features of the detection frame. That is, for each weight feature in the at least one weight feature corresponding to the initial features of the above-mentioned detection frame, the category corresponding to each weight feature is relatively similar to the category of the detection frame.

[0058] In one implementation of the disclosed embodiment, after the electronic device determines the similarity between the weight features corresponding to each of the multiple categories and the initial features of a detection frame, it can also determine the weight features corresponding to the X (X≥1) categories with the greatest similarity as at least one weight feature corresponding to the initial features of the detection frame.

[0059] Optionally, the electronic device may characterize the similarity between the features of the weight corresponding to each of the above categories and the initial features of a detection frame in the multiple detection frames in the form of cosine distance. Specifically, when the cosine distance between the initial features of the detection frame and the features of the weight corresponding to a certain category is less than or equal to a distance threshold, the electronic device may determine the features of the weight corresponding to the category as one of the at least one weight features corresponding to the initial features of the detection frame. Alternatively, the electronic device may also determine the cosine distance between the features of the weight corresponding to each category in the multiple categories and the initial features of the detection frame, and then determine the features of the weight corresponding to the X categories with the smallest cosine distance as the at least one weight feature corresponding to the initial features of the detection frame.

[0060] Optionally, the first classifier may be a 20,000-class classifier.

[0061] S102: The electronic device generates a target feature for each detection frame based on the initial feature of each detection frame in the multiple detection frames and at least one weight feature corresponding to the initial feature of each detection frame.

[0062] In combination with the description of the above embodiments, it should be understood that for a certain detection frame among the multiple detection frames, at least one weight feature corresponding to the initial feature of the detection frame is a weight feature corresponding to a category that is relatively similar to the category of the detection frame (specifically, the category of the object included in the detection frame).

[0063] In the disclosed embodiment, the electronic device generates the target features of each detection frame based on the initial features of each detection frame and at least one weight feature corresponding to the initial features of each detection frame, that is, the features of the category of each detection frame can be fused with the features of at least one category (specifically, a category in the image classification model that is relatively similar to the category of each detection frame). Since the image classification model includes a large number of categories, the electronic device can achieve the migration from the classification image to the category on the detection image without the need for a large amount of manual labeling of the training samples of the target detection model, and can accurately generate target features that can characterize the real data distribution of the category of each detection frame (specifically, the category of the object included in each detection frame).

[0064] In an optional implementation, for a certain detection frame among the above-mentioned multiple detection frames, the electronic device may perform mean fusion of the initial feature of the detection frame and at least one weight feature corresponding to the initial feature of the detection frame, that is, determine the average value of the initial feature of the detection frame and at least one weight feature corresponding to the initial feature of the detection frame, and then the electronic device determines the average value as the target feature of the detection frame.

[0065] S103: The electronic device trains the first object detection model based on the object features of each detection frame to generate a second object detection model.

[0066] It should be understood that the first target detection model is a target detection model in an initial state, and the second target detection model is a target detection model that has been trained.

[0067] Specifically, the electronic device may input the target features of each detection frame into the first object detection model to obtain a predicted classification result for each detection frame in the first object detection model. The electronic device may then determine the loss in the first object detection model based on the actual classification result of each detection frame, and update the parameters in the first object detection model based on the loss to generate the second object detection model.

[0068] The technical solution provided by the above embodiment can at least bring the following beneficial effects: As can be seen from S101-S103, the electronic device can determine the initial features of each detection frame in the multiple detection frames included in the detected image and at least one weight feature corresponding to each detection frame, and generate the target features of each detection frame based on the initial features of each detection frame and the at least one weight feature corresponding to each detection frame. Since the at least one weight feature corresponding to each detection frame is a feature of the weight included in the first classifier, a weight corresponds to a category in the first classifier, and the similarity between the at least one weight feature corresponding to a detection frame and the detection frame is greater than or equal to the similarity threshold, the electronic device can determine at least one category (i.e., the category corresponding to each of the at least one weight feature corresponding to the initial features of the detection frame) that is relatively similar to the category of the detection frame (specifically, the category of the object included in the detection frame) from the multiple categories included in the first classifier. The electronic device can then combine the features corresponding to the category of the detection frame (i.e., the initial features of the detection frame) with the features corresponding to the at least one category (i.e., at least one weight feature corresponding to the initial features of the detection frame), thereby realizing the migration of categories from the classified image to the detection image. This eliminates the need for a large amount of manual annotation of the training samples of the target detection model, and can accurately generate target features that can characterize the real data distribution of the category of each detection frame (specifically, the category of the object included in each detection frame). The electronic device can then train the first target detection model based on the target features of each detection frame to generate a second target detection model, thereby improving the training efficiency of the target detection model. Furthermore, the electronic device can quickly and accurately predict the categories of multiple objects included in each image to be detected based on the second target detection model, thereby improving the prediction efficiency of the target detection model.

[0069] Combine Figure 1 ,like Figure 2 As shown, in one implementation of the embodiment of the present disclosure, the above-mentioned first target detection model includes a first sub-network model and a second sub-network model, and the above-mentioned electronic device determines the initial features of each detection box in the multiple detection boxes included in the detected picture, which may specifically include S1011-S1014.

[0070] S1011. The electronic device obtains the detected image.

[0071] It should be understood that the detected picture is a training sample included in the training sample set of the above-mentioned first target detection model, and the number of the detected pictures can be one or more.

[0072] S1012: The electronic device inputs the detected image into the first sub-network model to perform feature recognition to obtain initial features of the detected image.

[0073] Optionally, the first sub-network model can be a convolutional neural network (CNN), and the CNN can specifically be ResNet50, ResNet101, ResNet200, ResNeXt101, ResNeSt101, ResNet18, MobileNet or SqueezeNet, etc.

[0074] Optionally, before the electronic device inputs the detected image into the first sub-network model, the detected image may be cropped. For example, the short side of the cropped detected image may be 600 pixels long, and the long side may be less than or equal to 1000 pixels.

[0075] S1013: The electronic device inputs the initial features of the detected image into the second sub-network model to obtain the position of each detection frame in the multiple detection frames and the size of each detection frame.

[0076] In one implementation of the disclosed embodiment, for a detection frame among the multiple detection frames, the position of the detection frame may be represented by the coordinates of the center point of the detection frame, and the size of the detection frame may be represented by the length and width of the detection frame.

[0077] Optionally, the second sub-network model may be a region proposal network (RPN).

[0078] S1014: The electronic device extracts initial features of each detection frame from initial features of the detected image based on the position and size of each detection frame.

[0079] It should be understood that the initial features of the detected image include the initial features of all content included in the detected image. In the disclosed embodiment, after obtaining the position and size of each detection frame in multiple detection frames, the electronic device can determine the position and size of a detection frame in the detected image based on the position and size of the detection frame, and then extract the initial features corresponding to the position and size of the detection frame in the detected image from the initial features of the detected image, that is, the initial features of the detection frame can be extracted.

[0080] The technical solution provided by the above embodiment can at least bring the following beneficial effects: As can be seen from S1011-S1014, after the electronic device obtains the detected image, it can input the detected image into the first sub-network model included in the first target detection model to obtain the initial features of the detected image; and the initial features of the detected image can be input into the second sub-network model included in the first target detection model to obtain the position of each detection frame in the multiple detection frames included in the detected image and the size of each detection frame; then the electronic device can extract the initial features of each detection frame from the initial features of the detected image based on the position of each detection frame and the size of each detection frame. In the embodiment of the present disclosure, the electronic device can determine the position and size of a detection frame in the detected image based on the position of a detection frame included in the detected image and the size of the detection frame, and then extract the initial features corresponding to the position and size of the detection frame in the detected image from the initial features of the detected image, that is, the initial features of the detection frame can be extracted. The initial features of each detection frame included in the detected image can be accurately and effectively extracted.

[0081] Combine Figure 2 ,like Figure 3 As shown, in one implementation of the embodiment of the present disclosure, the above-mentioned first classifier is a classifier included in the above-mentioned first image classification model, and the first image classification model includes a third sub-network model. The parameters in the third sub-network model are the same as the parameters in the above-mentioned first sub-network model. The training method of the target detection model provided by the embodiment of the present disclosure can also include S104-S106.

[0082] S104: The electronic device obtains a plurality of classified images.

[0083] It should be understood that the multiple classified images are training samples included in the training sample set of the first image classification model.

[0084] S105: The electronic device inputs the multiple classified images into the third sub-network model to obtain initial features of each of the multiple classified images.

[0085] It is understandable that the parameters in the third sub-network model are the same as those in the first sub-network model, indicating that the third sub-network model and the first sub-network model are the same network model. The third sub-network model and the first sub-network model can also be understood as twin networks.

[0086] Optionally, the third sub-network model may also be a CNN.

[0087] In an optional implementation, the electronic device may input the detected image and the multiple classified images into the same network model, that is, the first image classification model may include the first sub-network model. After obtaining the multiple classified images, the electronic device may input the multiple classified images into the first sub-network model to obtain initial features for each of the multiple classified images.

[0088] Optionally, before the electronic device inputs the multiple classified images into the third sub-network model, the multiple classified images may also be cropped. For example, the electronic device may crop each of the multiple classified images into an image of 224 pixels * 224 pixels in size.

[0089] It should be noted that the electronic device performs cropping processing on the detected images and the multiple classified images to ensure that the training samples in the same batch can be trained at the same time.

[0090] Optionally, in order to improve the speed and accuracy of model training, the electronic device can use the ImageNet dataset to pre-train the first sub-network model and the third sub-network model.

[0091] S106: The electronic device inputs the initial features of each classified image into the first classifier to obtain a predicted classification result of each classified image in the first classifier.

[0092] In conjunction with the description of the above embodiment, it should be understood that each of the multiple classification images may include an object, and the category of the classification image is the category of the object included in the classification image. In the disclosed embodiment, the predicted classification result of each classification image obtained by the electronic device in the first classifier is the predicted classification result of the object included in each classification image in the first classifier.

[0093] The technical solution provided by the above embodiment can at least bring the following beneficial effects: As can be seen from S104-S106, the electronic device can obtain multiple classified pictures, and input the multiple classified pictures into the third sub-network model (the parameters in the third sub-network model are the same as the parameters of the first sub-network model included in the first target detection model) to obtain the initial features of each classified picture in the multiple classified pictures; then the electronic device inputs the initial features of each classified picture into the first classifier (i.e., the classifier included in the first picture classification model) to obtain the predicted classification results of each classified picture in the first classifier. In the embodiment of the present disclosure, the electronic device can jointly train multiple classified pictures and detected pictures through a twin network (i.e., the first sub-network model and the third sub-network model), and can quickly determine the predicted classification results corresponding to each of the multiple classified pictures and the detected pictures. Furthermore, the electronic device can determine the losses in the first picture classification model and the first target detection model based on the predicted classification results corresponding to each of the multiple classified pictures and the detected pictures, so as to update the parameters in the first picture classification model and the first target detection model, thereby improving the efficiency of model training.

[0094] Combine Figure 1 ,like Figure 4 As shown, the training method of the target detection model provided by the embodiment of the present disclosure also includes S107.

[0095] S107 . The electronic device determines a first loss and a second loss.

[0096] Among them, the first loss is used to characterize the degree of inconsistency between the true classification result of each classified image in a plurality of classified images and the predicted classification result of each classified image in the above-mentioned first classifier, and the second loss is used to characterize the degree of spatial inconsistency of each detection frame in the above-mentioned plurality of detection frames, wherein the spatial inconsistency degree of a detection frame includes the degree of inconsistency between the true position of the detection frame and the predicted position of the detection frame in the first target detection model, and the degree of inconsistency between the true size of the detection frame and the predicted size of the detection frame in the first target detection model.

[0097] It should be understood that when the electronic device obtains the multiple classified images, it can also obtain the real classification results of the multiple classified images (which can also be understood as the real labels of the multiple classified images). The real classification result of a classified image is used to characterize the real category of the object included in the classified image. Similarly, when the electronic device obtains the above-mentioned detected image, it can also obtain the real position of each detection frame in the multiple detection frames included in the detected image and the real size of each detection frame. Specifically, the real position of each detection frame and the real size of each detection frame can be reflected (or marked) in the detected image.

[0098] In combination with the description of the above embodiment, it should be understood that the electronic device inputs the multiple classified pictures into the above-mentioned third network sub-model to obtain the initial features of each classified picture in the multiple classified pictures, and then inputs the initial features of each classified picture into the first classifier, so as to obtain the predicted classification results of each classified picture in the first classifier. The predicted classification results of each classified picture in the first classifier can also be understood as the predicted classification results of each classified picture in the above-mentioned first picture classification model.

[0099] In an optional implementation, for a certain classified picture among the multiple classified pictures, the electronic device can define the true classification result of the classified picture as 1, and determine the score (for example, 0.5) of the true classification result predicted for the classified picture in the first classifier. The electronic device can then determine that the loss corresponding to the classified picture is the difference between the true classification result and the score (i.e., 1-0.5=0.5), and then the electronic device can determine the sum of the losses corresponding to each classified picture in the multiple classified pictures as the above-mentioned first loss.

[0100] In another optional implementation, the electronic device may also determine the first loss using a cross entropy function. For example, the electronic device may determine that the loss corresponding to the classified image is -log0.5.

[0101] It is understood that the first object detection model includes the second sub-network model. For a particular detection box among the multiple detection boxes, the predicted position of the detection box in the first object detection model is the position of the detection box obtained after the electronic device inputs the initial features of the detected image into the second sub-network model. Similarly, the predicted size of the detection box in the first object detection model is the size of the detection box obtained after the electronic device inputs the initial features of the detected image into the second sub-network model.

[0102] In one implementation of the disclosed embodiment, the predicted position of a detection frame in the first object detection model and the actual position of the detection frame can be represented in the form of coordinates, and the predicted size of the detection frame in the first object detection model and the actual size of the detection frame can be represented in the form of length and width. The electronic device can determine whether the spatial loss corresponding to the detection frame satisfies the following formula:

[0103] L1=|x1-x2|+|y1-y2|+|w1-w2|+|h1-h2|

[0104] Among them, L1 represents the spatial loss corresponding to the detection box, x1 represents the horizontal coordinate of the predicted position of the detection box in the first target detection model, y1 represents the vertical coordinate of the predicted position, w1 represents the length of the detection box in the predicted size in the first target detection model, h1 represents the width in the predicted size, x2 represents the horizontal coordinate of the actual position of the detection box, y2 represents the vertical coordinate of the actual position, w2 represents the length of the detection box in the actual size, and h2 represents the width in the actual size.

[0105] Optionally, the weight of the first loss may be 0.001, and the weight of the second loss may be 1.

[0106] Continue as Figure 4 As shown, the electronic device trains the first target detection model based on the target features of each detection frame to generate a second target detection model, which may specifically include S1031-S1034.

[0107] S1031: The electronic device inputs the target features of each detection frame into the second classifier to obtain a predicted classification result of each detection frame in the second classifier.

[0108] Among them, the second classifier is the classifier included in the above-mentioned first target detection model.

[0109] In the embodiment of the present disclosure, the number of categories included in the second classifier may be smaller than the number of categories included in the first classifier. For example, the second classifier may be an 80-category classifier.

[0110] S1032: The electronic device determines the third loss and the fourth loss.

[0111] Among them, the third loss is used to characterize the degree of inconsistency between the true classification result of each of the above-mentioned detection frames and the predicted classification result of each detection frame in the second classifier, and the fourth loss is used to characterize the degree of inconsistency between the true result of whether each detection frame includes an object and the result of whether each detection frame is predicted to include an object in the second classifier.

[0112] In combination with the description of the above embodiments, it should be understood that when the electronic device obtains the above-mentioned detected picture, it can also obtain the true classification result of each detection frame in the multiple detection frames included in the detected picture, that is, the true category of the object (or target) included in each detection frame.

[0113] It should be noted that the explanation of the electronic device determining the third loss is the same or similar to the description of the electronic device determining the above-mentioned first loss, and will not be repeated here.

[0114] It is understood that when the electronic device obtains the above-mentioned detected image, it can also obtain the actual result of whether each detection frame includes an object. Specifically, when the actual classification result of a detection frame is a certain object (or target), the electronic device can determine that the detection frame includes the object. Conversely, when the actual classification result of the detection frame is background (i.e., not an object), the electronic device can determine that the detection frame does not include an object.

[0115] For one of the multiple detection frames mentioned above, the result of the detection frame being predicted as including an object in the second classifier is the sum of the probabilities of the detection frame being predicted as each category in the second classifier, and the result of the detection frame being predicted as not including an object in the second classifier is the probability of the detection frame being predicted as the background in the second classifier.

[0116] S1033: The electronic device determines a target loss based on the first loss, the second loss, the third loss, and the fourth loss.

[0117] In an optional implementation, the electronic device may determine the sum of the first loss, the second loss, the third loss, and the fourth loss as the target loss.

[0118] S1034. The electronic device updates parameters in the first target detection model based on the target loss to generate a second target detection model.

[0119] It should be understood that the electronic device updates the parameters in the first target detection model, specifically updating the parameters in the second classifier, the parameters in the first sub-network model, and the parameters in the second sub-network model.

[0120] In an optional implementation, the electronic device may determine a preset gradient based on the target loss using stochastic gradient descent (SGD), and then update the parameters in the first target detection model based on the preset gradient to obtain the second target detection model.

[0121] Optionally, during the training process of the first target detection model, the learning rate can be set to 0.002, and the entire training process of the second target detection model can be trained for a total of 30,000 rounds, wherein the learning rate can be reduced by 10 times in the 24,000th round, that is, the learning rate is updated to 0.0002.

[0122] In one implementation of the embodiment of the present invention, the electronic device may also update the parameters of the first image classification model based on the target loss to generate a second image classification model, where the second image classification model can be understood as a trained image classification model. The electronic device updates the parameters of the first image classification model by specifically updating the parameters of the first classifier and the parameters of the third sub-network model.

[0123] The technical solution provided by the above embodiment can at least bring the following beneficial effects: As can be seen from S107, S1031-S1034, the electronic device can determine a first loss, a second loss, a third loss and a fourth loss, wherein the first loss is used to characterize the degree of inconsistency between the actual classification result of each classified image in a plurality of classified images and the predicted classification result of each classified image in the first classifier, the second loss is used to characterize the degree of spatial inconsistency of each detection frame in a plurality of detection frames included in the detected image (including the degree of inconsistency between the actual position of a detection frame and the predicted position of the detection frame in the first target detection model, and the degree of inconsistency between the actual size of the detection frame and the predicted size of the detection frame in the target detection model), the third loss is used to characterize the degree of inconsistency between the actual classification result of each detection frame and the predicted classification result of each detection frame in the second classifier, and the fourth loss is used to characterize the degree of inconsistency between the actual result of whether each detection frame includes an object and the result of each detection frame being predicted as including an object in the second classifier. The electronic device can then determine a target loss based on the first loss, the second loss, the third loss, and the fourth loss, and based on the target loss, update the parameters in the first target detection model to generate a second target detection model. In the disclosed embodiment, the electronic device can determine each loss that exists in the first target detection model and the first image classification model during the training process, and update the parameters in the first target detection model and the first image classification model based on each loss, thereby accurately and effectively training the target detection model to generate a target detection model with higher prediction accuracy.

[0124] Combine Figure 1 ,like Figure 5 As shown, the training method of the target detection model provided by the embodiment of the present disclosure may also include S108-S110.

[0125] S108: The electronic device determines the target feature of the first detection frame as a preset mean value.

[0126] The first detection frame is one of the multiple detection frames mentioned above.

[0127] In conjunction with the description of the above embodiment, it should be understood that the electronic device can generate a target feature for each detection frame based on the initial features of each detection frame in the multiple detection frames and at least one weight corresponding to the initial features of each detection frame. Then, for a detection frame in the multiple detection frames (e.g., the first detection frame), the electronic device can determine the target feature of the first detection frame as the preset mean.

[0128] S109: The electronic device generates a plurality of extended features of the first detection frame based on a preset mean and a preset standard deviation.

[0129] In an optional implementation, the electronic device may generate multiple extended features of the first detection frame based on the preset mean and the preset standard deviation and using a Gaussian function.

[0130] Specifically, the electronic device may determine that the multiple extended features of the first detection frame satisfy the following formula:

[0131] I g =Gaussian(I t ,σ)

[0132] Among them, I g Represents multiple extended features of the first detection frame, I t represents the preset mean, σ represents the preset standard deviation, and Gaussian represents the Gaussian function.

[0133] Exemplarily, the electronic device may set the preset standard deviation to 1.

[0134] S110: The electronic device trains a first object detection model based on multiple extended features of each detection box in the multiple detection boxes.

[0135] It should be understood that the electronic device can generate more features that can characterize the category of each detection frame in the multiple detection frames (i.e., multiple extended features for each detection frame) based on the above-mentioned preset mean and preset standard deviation, which can improve the richness of the features to be trained (i.e., the features used to train the first object detection model). Furthermore, when the electronic device trains the first object detection model based on the multiple extended features of each detection frame, it can train an object detection model with higher prediction accuracy.

[0136] It should be noted that the explanation of the electronic device training the first target detection model based on the multiple extended features of each detection frame in the multiple detection frames is the same or similar to the description of the above-mentioned electronic device training the first target detection model based on the target features of each detection frame, and will not be repeated here.

[0137] The technical solution provided by the above embodiment can at least bring about the following beneficial effects: As can be seen from S108-S110, the electronic device can determine the target feature of a detection frame (e.g., the first detection frame) among the multiple detection frames as a preset mean, and generate multiple extended features of the first detection frame based on the preset mean and a preset standard deviation. Because the multiple extended features of each detection frame among the multiple detection frames can improve the richness during model training, the electronic device can train a target detection model with higher prediction accuracy when training the first target detection model based on the multiple extended features of each detection frame.

[0138] like Figure 6 As shown, the target detection method provided by the embodiment of the present disclosure may include S201-S202.

[0139] S201: The electronic device obtains a picture to be detected.

[0140] It should be understood that the image to be detected is a detection image without a real classification result.

[0141] S202: The electronic device inputs the image to be detected into a second object detection model to obtain a category of at least one detection frame included in the image to be detected.

[0142] Among them, the second target detection model is trained based on the training method of the target detection model provided in the above-mentioned embodiment of the present disclosure.

[0143] Specifically, the second target detection model is generated by training the first target detection model based on the target features of each detection frame in the multiple detection frames included in the detected image, and the target features of each detection frame are generated based on the initial features of each detection frame and at least one weight feature corresponding to the initial features of each detection frame, wherein the similarity between each weight feature of the at least one weight feature corresponding to the initial feature of a detection frame and the initial feature of the detection frame is greater than or equal to a similarity threshold, and the at least one weight feature corresponding to the initial feature of each detection frame is a feature of the weight included in the first classifier, and a weight corresponds to a category in the first classifier, and the first category is the classifier included in the first image classification model.

[0144] In combination with the description of the above embodiments, it should be understood that the first target detection model is a target detection model in an initial state, the second target detection model is a target detection model that has been trained, the first image classification model is an image classification model in an initial state, and the category of each of the at least one detection frame included in the image to be detected is the category of the object (or target) included in the at least one detection frame.

[0145] It is understood that the second image classification model is a trained image classification model. The electronic device can obtain a certain image to be classified and input the image to be classified into the second image classification model to obtain the category of the image to be classified (specifically, the object included in the image to be classified).

[0146] It should be noted that the specific process of the electronic device generating the target features of each detection frame in the multiple detection frames, and the specific process of the electronic device training the first target detection model based on the target features of each detection frame to generate the second target detection model can be found in the description of the above embodiments and will not be repeated here.

[0147] The technical solution provided by the above embodiment can at least bring the following beneficial effects: As can be seen from S201-S202, the electronic device can obtain the picture to be detected, and input the picture to be detected into the second target detection model to obtain the category of at least one detection frame included in the picture to be detected. In the embodiment of the present disclosure, since the similarity between the initial feature of a detection frame and the at least one weight feature corresponding to the initial feature of the detection frame is greater than or equal to the similarity threshold, it means that the electronic device can determine at least one category that is relatively similar to the category of the detection frame from the multiple categories included in the first classifier (that is, the category corresponding to the at least one weight feature corresponding to the initial feature of the detection frame). The target feature of the detection frame can realize the migration of the category from the classification picture to the detection picture, without the need for a large amount of manual annotation of the training samples of the target detection model, and can accurately characterize the category of each detection frame, that is, the second target detection model is a target detection model with high prediction speed and high prediction accuracy. In this way, the electronic device can accurately determine the category of at least one detection frame included in the picture to be detected based on the second target detection model, which can improve the efficiency of target detection.

[0148] Combine Figure 6 ,like Figure 7 As shown, the above-mentioned second target detection model includes a first sub-network model, a second sub-network model and a second classifier. The above-mentioned electronic device inputs the picture to be detected into the second target detection model to obtain the category of at least one detection box included in the picture to be detected, which can specifically include S2021-S2024.

[0149] S2021. The electronic device inputs the image to be detected into the first sub-network model, performs feature recognition, and obtains initial features of the image to be detected.

[0150] S2022: The electronic device inputs the initial features of the image to be detected into the second sub-network model to obtain the position of at least one detection frame and the size of at least one detection frame.

[0151] S2023. The electronic device extracts the initial features of at least one detection frame from the initial features of the image to be detected based on the position of the at least one detection frame and the size of the at least one detection frame.

[0152] S2024. The electronic device inputs the initial features of at least one detection frame into a second classifier to obtain a category of at least one detection frame.

[0153] It should be noted that the explanations in the above S2021-S2024 can be found in the description in the above embodiments and will not be repeated here.

[0154] The technical solution provided by the above embodiment can at least bring the following beneficial effects: It can be seen from S2021-S2024 that the electronic device can input the picture to be detected into the first sub-network model included in the second target detection model to obtain the initial features of the picture to be detected; and the initial features of the picture to be detected are input into the second sub-network model included in the second target detection model to obtain the respective positions of at least one detection frame included in the picture to be detected and the respective sizes of the at least one detection frame; then the electronic device can extract the respective initial features of the at least one detection frame from the initial features of the picture to be detected based on the respective positions of the at least one detection frame and the respective sizes of the at least one detection frame, and input the respective initial features of the at least one detection frame into the second classifier included in the second target detection model to obtain the respective categories of the at least one detection frame. In the embodiment of the present disclosure, the electronic device can accurately and effectively determine the respective categories of the at least one detection frame included in the picture to be detected, that is, it can accurately and effectively determine the categories of the objects included in the at least one detection frame.

[0155] It is understandable that, in actual implementation, the electronic device described in the embodiment of the present disclosure may include one or more hardware structures and / or software modules for implementing the training method of the aforementioned corresponding target detection model and the target detection method, and these execution hardware structures and / or software modules may constitute an electronic device. It should be easy for those skilled in the art to realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0156] Based on this understanding, the present disclosure also provides a training device for a target detection model. Figure 8FIG. 1 shows a schematic diagram of the structure of the training device of the target detection model provided by the embodiment of the present disclosure. Figure 8 As shown, the target detection model training device 10 may include: a determination module 101 and a processing module 102.

[0157] The determination module 101 is configured to determine the initial features of each detection frame in the multiple detection frames included in the detected image and at least one weight feature corresponding to the initial features of each detection frame, wherein the similarity between each weight feature of the at least one weight feature corresponding to the initial features of a detection frame and the initial features of the detection frame is greater than or equal to a similarity threshold, and the at least one weight feature corresponding to the initial features of each detection frame is a feature of the weight included in the first classifier, and one weight corresponds to a category in the first classifier.

[0158] The processing module 102 is configured to generate a target feature of each detection frame based on the initial feature of each detection frame in the multiple detection frames and at least one weight feature corresponding to the initial feature of each detection frame.

[0159] The processing module 102 is further configured to train the first object detection model based on the object features of each detection box to generate a second object detection model.

[0160] Optionally, the first target detection model includes a first sub-network model and a second sub-network model, and the training device 10 of the target detection model further includes an acquisition module 103.

[0161] The acquisition module 103 is configured to acquire the detected image.

[0162] The processing module 102 is specifically configured to input the detected image into the first sub-network model to perform feature recognition to obtain initial features of the detected image.

[0163] The processing module 102 is further configured to input the initial features of the detected image into the second sub-network model to obtain the position of each detection frame in the multiple detection frames and the size of each detection frame.

[0164] The processing module 102 is further configured to extract the initial features of each detection frame from the initial features of the detected image based on the position of each detection frame and the size of each detection frame.

[0165] Optionally, the above-mentioned first classifier is a classifier included in a first image classification model, and the first image classification model includes a third sub-network model, and the parameters in the third sub-network model are the same as the parameters in the first sub-network model.

[0166] The acquisition module 103 is further configured to acquire a plurality of classified images.

[0167] The processing module 102 is further configured to input the multiple classified images into the third sub-network model to obtain initial features of each classified image in the multiple classified images;

[0168] The processing module 102 is further configured to input the initial features of each classified image into the first classifier to obtain a predicted classification result of each classified image in the first classifier.

[0169] Optionally, the determination module 101 is further configured to determine a first loss and a second loss, the first loss being used to characterize the degree of inconsistency between the actual classification result of each classified image in a plurality of classified images and the predicted classification result of each classified image in the first classifier, and the second loss being used to characterize the degree of spatial inconsistency of each detection frame in the plurality of detection frames, wherein the spatial inconsistency degree of a detection frame includes the degree of inconsistency between the actual position of the detection frame and the predicted position of the detection frame in the first target detection model, and the degree of inconsistency between the actual size of the detection frame and the predicted size of the detection frame in the first target detection model.

[0170] The processing module 102 is specifically configured to input the target features of each detection box into a second classifier to obtain a predicted classification result of each detection box in the second classifier, where the second classifier is a classifier included in the first target detection model.

[0171] The determination module 101 is specifically configured to determine a third loss and a fourth loss, wherein the third loss is used to characterize the degree of inconsistency between the true classification result of each detection frame and the predicted classification result of each detection frame in the second classifier, and the fourth loss is used to characterize the degree of inconsistency between the true result of whether each detection frame includes an object and the result of whether each detection frame is predicted in the second classifier as including an object.

[0172] The determination module 101 is further configured to determine a target loss according to the first loss, the second loss, the third loss, and the fourth loss.

[0173] The processing module 102 is further configured to update the parameters in the first target detection model based on the target loss to generate the second target detection model.

[0174] Optionally, the determination module 101 is further configured to determine the target feature of the first detection frame as a preset mean value, and the first detection frame is one of the multiple detection frames.

[0175] The processing module 102 is further configured to generate a plurality of extended features of the first detection frame based on the preset mean and the preset standard deviation.

[0176] The processing module 102 is further configured to train the first object detection model based on multiple extended features of each detection box in the multiple detection boxes.

[0177] As described above, the embodiment of the present disclosure can divide the functional modules of the training device of the target detection model according to the above method example. Among them, the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. In addition, it should be noted that the division of modules in the embodiment of the present disclosure is schematic and is only a logical functional division. There may be other division methods in actual implementation. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module.

[0178] Regarding the training device for the target detection model in the above embodiment, the specific manner in which each module performs operations and the beneficial effects it has have been described in detail in the aforementioned method embodiment and will not be repeated here.

[0179] Figure 9 Schematic diagram of another target detection model training device provided by the present disclosure. Figure 9 The target detection model training device 20 may include at least one processor 201 and a memory 203 for storing processor-executable instructions. The processor 201 is configured to execute instructions in the memory 203 to implement the target detection model training method in the above embodiment.

[0180] In addition, the target detection model training device 20 may further include a communication bus 202 and at least one communication interface 204 .

[0181] The processor 201 may be a central processing unit (CPU), a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of the program of the disclosed solution.

[0182] The communication bus 202 may include a pathway for transmitting information between the aforementioned components.

[0183] The communication interface 204 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0184] The memory 203 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processing unit via a bus. The memory may also be integrated with the processing unit.

[0185] The memory 203 is used to store instructions for executing the solution of the present disclosure, and the execution is controlled by the processor 201. The processor 201 is used to execute the instructions stored in the memory 203, thereby realizing the functions of the method of the present disclosure.

[0186] In a specific implementation, as an embodiment, the processor 201 may include one or more CPUs, such as Figure 9 CPU0 and CPU1 in.

[0187] In a specific implementation, as an embodiment, the target detection model training device 20 may include multiple processors, such as Figure 9 201 and processor 207 in FIG. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0188] In a specific implementation, as an embodiment, the training device 20 of the target detection model may further include an output device 205 and an input device 206. The output device 205 communicates with the processor 201 and can display information in a variety of ways. For example, the output device 205 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 206 communicates with the processor 201 and can accept user input in a variety of ways. For example, the input device 206 may be a mouse, a keyboard, a touch screen device, or a sensor device.

[0189] Figure 10 This is a structural example diagram of a target detection device provided by the present disclosure. Figure 10 As shown, the target detection device 30 may include: According to the fourth aspect of the embodiment of the present disclosure, a target detection device is provided. The target detection device 30 may include: an acquisition module 301 and a processing module 302.

[0190] The acquisition module 301 is configured to acquire a picture to be detected.

[0191] The processing module 302 is configured to input the image to be detected into a second target detection model to obtain the category of at least one detection box included in the image to be detected. The second target detection model is trained based on the training method of the target detection model provided in the above-mentioned embodiment of the present disclosure.

[0192] Optionally, the second target detection model includes a first sub-network model, a second sub-network model and a second classifier.

[0193] The processing module 302 is specifically configured to input the image to be detected into the first sub-network model, perform feature recognition, and obtain initial features of the image to be detected.

[0194] The processing module 302 is further configured to input the initial features of the image to be detected into the second sub-network model to obtain the position of each of the at least one detection box and the size of each of the at least one detection box.

[0195] The processing module 302 is further configured to extract the initial features of the at least one detection frame from the initial features of the image to be detected based on the position of the at least one detection frame and the size of the at least one detection frame.

[0196] The processing module 302 is further configured to input the initial features of the at least one detection box into the second classifier to obtain the category of the at least one detection box.

[0197] Figure 11 : is a schematic diagram of the structure of another target detection device provided by the present disclosure, which can be the above-mentioned electronic device. Figure 11 The target detection device 40 may include at least one processor 401 and a memory 403 for storing processor-executable instructions. The processor 401 is configured to execute the instructions in the memory 403 to implement the target detection method in the above embodiment.

[0198] In addition, the target detection device 40 may further include a communication bus 402 and at least one communication interface 404 .

[0199] The processor 401 may be a CPU, a microprocessor unit, an ASIC, or one or more integrated circuits for controlling the execution of the program of the disclosed solution.

[0200] The communication bus 402 may include a pathway for transmitting information between the aforementioned components.

[0201] The communication interface 404 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, RAN, WLAN, etc.

[0202] The memory 403 may be a ROM or other type of static storage device capable of storing static information and instructions, a RAM or other type of dynamic storage device capable of storing information and instructions, an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory may be independent and connected to the processing unit via a bus. The memory may also be integrated with the processing unit.

[0203] The memory 403 is used to store instructions for executing the solution of the present disclosure, and the execution is controlled by the processor 401. The processor 401 is used to execute the instructions stored in the memory 403, thereby realizing the functions of the method of the present disclosure.

[0204] In a specific implementation, as an embodiment, the processor 401 may include one or more CPUs, such as Figure 11 CPU0 and CPU1 in.

[0205] In a specific implementation, as an embodiment, the target detection device 40 may include multiple processors, such as Figure 11 4 and 5. The processors 401 and 407 are shown in FIG. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0206] In a specific implementation, as an embodiment, the target detection apparatus 40 may further include an output device 405 and an input device 406. The output device 405 communicates with the processor 401 and can display information in a variety of ways. For example, the output device 405 can be an LCD, LED display device, CRT display device, or projector. The input device 406 communicates with the processor 401 and can receive user input in a variety of ways. For example, the input device 406 can be a mouse, keyboard, touch screen device, or sensor device.

[0207] Those skilled in the art will understand that Figure 9 The structure shown in does not constitute a limitation on the training device 20 of the target detection model, and Figure 11 The structure shown in the figure does not constitute a limitation on the target detection device 40. It may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0208] In addition, the present disclosure also provides a computer-readable storage medium, including instructions, which, when executed by an electronic device, enable the electronic device to execute the training method of the target detection model provided in the above embodiment, or execute the target detection method provided in the above embodiment.

[0209] In addition, the present disclosure also provides a computer program product, including instructions, which, when executed by an electronic device, enable the electronic device to execute the target detection model training method provided in the above embodiment, or execute the target detection method provided in the above embodiment.

[0210] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A method for training a target detection model, characterized in that: include: Determining an initial feature of each detection frame in a plurality of detection frames included in the detected image and at least one weight feature corresponding to the initial feature of each detection frame, wherein a similarity between each weight feature of the at least one weight feature corresponding to the initial feature of a detection frame and the initial feature of the detection frame is greater than or equal to a similarity threshold, and the at least one weight feature corresponding to the initial feature of each detection frame is a feature of a weight included in a first classifier, and one weight corresponds to a category in the first classifier; generating a target feature for each detection frame based on an initial feature of each detection frame and at least one weight feature corresponding to the initial feature of each detection frame; Based on the target features of each detection box, the first target detection model is trained to generate a second target detection model.

2. The target detection model training method according to claim 1, characterized in that: The first target detection model includes a first sub-network model and a second sub-network model, and determining the initial features of each of the multiple detection boxes included in the detected image includes: Obtaining the detected image; Inputting the detected image into the first sub-network model to perform feature recognition to obtain initial features of the detected image; Inputting the initial features of the detected image into the second sub-network model to obtain the position of each detection box in the multiple detection boxes and the size of each detection box; Based on the position of each detection frame and the size of each detection frame, the initial features of each detection frame are extracted from the initial features of the detected image.

3. The target detection model training method according to claim 2, characterized in that: The first classifier is a classifier included in a first image classification model, the first image classification model includes a third sub-network model, and parameters in the third sub-network model are the same as parameters in the first sub-network model. The method further includes: Get multiple classified images; Inputting the multiple classified images into the third sub-network model to obtain initial features of each of the multiple classified images; The initial features of each classified image are input into the first classifier to obtain a predicted classification result of each classified image in the first classifier.

4. The target detection model training method according to claim 3, characterized in that: The method further comprises: Determine a first loss and a second loss, where the first loss is used to characterize a degree of inconsistency between a true classification result of each of a plurality of classified images and a predicted classification result of each classified image in the first classifier, and the second loss is used to characterize a degree of spatial inconsistency of each of the plurality of detection frames, wherein the spatial inconsistency of a detection frame includes a degree of inconsistency between a true position of the detection frame and a predicted position of the detection frame in the first object detection model, and a degree of inconsistency between a true size of the detection frame and a predicted size of the detection frame in the first object detection model; The training of the first target detection model based on the target features of each detection frame to generate a second target detection model includes: Inputting the target feature of each detection box into a second classifier to obtain a predicted classification result of each detection box in the second classifier, where the second classifier is a classifier included in the first object detection model; determining a third loss and a fourth loss, wherein the third loss is used to characterize a degree of inconsistency between a true classification result of each detection frame and a predicted classification result of each detection frame in the second classifier, and the fourth loss is used to characterize a degree of inconsistency between a true result of whether each detection frame includes an object and a result predicted by the second classifier as to whether each detection frame includes an object; determining a target loss according to the first loss, the second loss, the third loss, and the fourth loss; Based on the target loss, parameters in the first target detection model are updated to generate the second target detection model.

5. The method for training a target detection model according to any one of claims 1 to 4, characterized in that: The method further comprises: Determining a target feature of a first detection frame as a preset mean value, where the first detection frame is one of the multiple detection frames; Generating a plurality of extended features of the first detection frame based on the preset mean and the preset standard deviation; The first object detection model is trained based on multiple extended features of each detection box in the multiple detection boxes.

6. A target detection method, characterized in that: include: Get the image to be detected; The image to be detected is input into a second target detection model to obtain the category of at least one detection frame included in the image to be detected, and the second target detection model is trained based on the training method of the target detection model according to any one of claims 1-5.

7. The target detection method according to claim 6, characterized in that: The second object detection model includes a first sub-network model, a second sub-network model, and a second classifier. Inputting the image to be detected into the second object detection model to obtain the category of at least one detection box included in the image to be detected includes: Inputting the image to be detected into the first sub-network model to perform feature recognition to obtain initial features of the image to be detected; Inputting the initial features of the image to be detected into the second sub-network model to obtain the position of each of the at least one detection frame and the size of each of the at least one detection frame; Extracting initial features of each of the at least one detection frame from initial features of the image to be detected based on the position of each of the at least one detection frame and the size of each of the at least one detection frame; The initial features of each of the at least one detection frame are input into the second classifier to obtain the category of each of the at least one detection frame.

8. A training device for a target detection model, characterized in that: include: Identify modules and process modules; The determination module is configured to determine an initial feature of each detection frame included in a plurality of detection frames in the detected image and at least one weight feature corresponding to the initial feature of each detection frame, wherein a similarity between each weight feature of the at least one weight feature corresponding to the initial feature of a detection frame and the initial feature of the detection frame is greater than or equal to a similarity threshold, and the at least one weight feature corresponding to the initial feature of each detection frame is a feature of a weight included in a first classifier, and one weight corresponds to a category in the first classifier; The processing module is configured to generate a target feature of each detection frame based on an initial feature of each detection frame and at least one weight feature corresponding to the initial feature of each detection frame; The processing module is further configured to train the first target detection model based on the target features of each detection box to generate a second target detection model.

9. The target detection model training device according to claim 8, characterized in that: The first target detection model includes a first sub-network model and a second sub-network model, and the training device of the target detection model further includes an acquisition module; The acquisition module is configured to acquire the detected image; The processing module is specifically configured to input the detected image into the first sub-network model to perform feature recognition to obtain initial features of the detected image; The processing module is further configured to input the initial features of the detected image into the second sub-network model to obtain the position of each detection box in the multiple detection boxes and the size of each detection box; The processing module is further configured to extract the initial features of each detection frame from the initial features of the detected image based on the position of each detection frame and the size of each detection frame.

10. The target detection model training device according to claim 9, characterized in that: The first classifier is a classifier included in a first image classification model, the first image classification model includes a third sub-network model, and the parameters in the third sub-network model are the same as the parameters in the first sub-network model; The acquisition module is further configured to acquire a plurality of classified images; The processing module is further configured to input the multiple classified images into the third sub-network model to obtain initial features of each classified image in the multiple classified images; The processing module is further configured to input the initial features of each classified image into the first classifier to obtain a predicted classification result of each classified image in the first classifier.

11. The target detection model training device according to claim 10, characterized in that: The determination module is further configured to determine a first loss and a second loss, wherein the first loss is used to characterize the degree of inconsistency between a true classification result of each of the multiple classified images and a predicted classification result of each classified image in the first classifier, and the second loss is used to characterize the degree of spatial inconsistency of each of the multiple detection frames, wherein the spatial inconsistency of a detection frame includes the degree of inconsistency between the true position of the detection frame and the predicted position of the detection frame in the first object detection model, and the degree of inconsistency between the true size of the detection frame and the predicted size of the detection frame in the first object detection model; The processing module is specifically configured to input the target feature of each detection box into a second classifier to obtain a predicted classification result of each detection box in the second classifier, where the second classifier is a classifier included in the first object detection model; The determination module is specifically configured to determine a third loss and a fourth loss, wherein the third loss is used to represent a degree of inconsistency between a true classification result of each detection frame and a predicted classification result of each detection frame in the second classifier, and the fourth loss is used to represent a degree of inconsistency between a true result of whether each detection frame includes an object and a result predicted by the second classifier as whether each detection frame includes an object; The determining module is further configured to determine a target loss according to the first loss, the second loss, the third loss, and the fourth loss; The processing module is further configured to update parameters in the first target detection model based on the target loss to generate the second target detection model.

12. The training device for the target detection model according to any one of claims 8 to 11, characterized in that: The determining module is further configured to determine a target feature of a first detection frame as a preset mean value, the first detection frame being one of the multiple detection frames; The processing module is further configured to generate a plurality of extended features of the first detection frame based on the preset mean and the preset standard deviation; The processing module is further configured to train the first object detection model based on multiple extended features of each detection box in the multiple detection boxes.

13. A target detection device, characterized in that: include: Acquisition module and processing module; The acquisition module is configured to acquire the image to be detected; The processing module is configured to input the image to be detected into a second target detection model to obtain the category of at least one detection box included in the image to be detected, and the second target detection model is trained based on the training method of the target detection model according to any one of claims 1-4.

14. The target detection device according to claim 13, wherein: The second target detection model includes a first sub-network model, a second sub-network model and a second classifier; The processing module is specifically configured to input the image to be detected into the first sub-network model, perform feature recognition, and obtain initial features of the image to be detected; The processing module is further configured to input the initial features of the image to be detected into the second sub-network model to obtain the position of each of the at least one detection boxes and the size of each of the at least one detection boxes; The processing module is further configured to extract the initial features of the at least one detection frame from the initial features of the image to be detected based on the position of the at least one detection frame and the size of the at least one detection frame; The processing module is further configured to input the initial features of each of the at least one detection box into the second classifier to obtain the category of each of the at least one detection box.

15. An electronic device, characterized in that: The electronic device comprises: processor; a memory configured to store instructions executable by the processor; The processor is configured to execute the instructions to implement the target detection model training method according to any one of claims 1 to 5, or to implement the target detection method according to claim 6 or 7.

16. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of the target detection model as described in any one of claims 1 to 5, or execute the target detection method as described in claim 6 or 7.

17. A computer program product, characterized in that The computer program product includes computer instructions, which, when executed on a processor of an electronic device, cause the electronic device to execute the target detection model training method described in any one of claims 1 to 5, or to execute the target detection method described in claim 6 or 7.

Images