Image recognition model training method, device, electronic device and storage medium
By predefining the correspondence between small and large categories of objects, training the image recognition model and calculating classification losses, the problem of limited computing resources and rapid adaptation to changes in driving strategies is solved, improving recognition accuracy and saving resources.
Patent Information
- Application Number
- CN202111080066.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-15
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-09-15
AI Technical Summary
When training image recognition models, the prior art has limited computing resources and is difficult to quickly adapt to changes in vehicle driving strategies, or it requires a lot of computing resources to meet driving strategies.
By pre-defined the correspondence between small classes and large classes of objects, the correspondence between small classes and large classes is used to train the image recognition model, calculate classification losses and update model parameters, and reduce the demand for computing resources.
With limited computing resources, the accuracy of object recognition is improved, and errors that identify objects belonging to a major category as another major category are reduced. It can meet the needs of vehicle driving strategies and save computing resources.
Smart Images

Figure CN113887581B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a training method, device, electronic device and storage medium for an image recognition model. Background Art
[0002] The vision-based autonomous driving solution needs to identify the images collected by the vehicle, including the size, location, and category of the objects in the image. Among them, the recognition result of the object category is an important factor in determining the vehicle's driving strategy. Therefore, how to identify the object category in the image is a problem that the autonomous driving solution needs to solve. Summary of the invention
[0003] The embodiments of the present application disclose a method, device, electronic device and storage medium for training an image recognition model, which can train an image recognition model that can meet the requirements of vehicle driving strategies under the premise of limited computing resources.
[0004] An embodiment of the present application discloses a training method for an image recognition model, the method comprising: performing sub-category recognition on a first object in a sample image through an image recognition model to be trained to obtain a sub-category prediction result of the first object; the sample image carries major category reference information of the first object; based on the correspondence between sub-categories and major categories, determining the target major category to which the sub-category prediction result belongs to obtain a major category prediction result; the correspondence comprises N major categories for object classification, each major category comprises M sub-categories, N is greater than or equal to 2, M is greater than or equal to 1, and sub-categories in different major categories correspond to different vehicle driving strategies; calculating the classification loss according to the major category reference information of the first object and the major category prediction result; and updating the parameters in the image recognition model according to the classification loss.
[0005] In one embodiment, the major category corresponding to the major category prediction result includes at least two minor categories; the sample image also carries minor category reference information of the first object; and the calculation of classification loss based on the major category reference information of the first object and the major category prediction result includes: calculating the major category loss based on the major category reference information of the first object and the major category prediction result; calculating the minor category loss based on the minor category reference information of the first object and the minor category prediction result; and calculating the classification loss based on the major category loss and the minor category loss.
[0006] In one embodiment, the calculating of the classification loss based on the major category loss and the minor category loss includes: performing weighted sum calculation based on a first weight corresponding to the major category and the major category loss, and a second weight corresponding to the minor category and the minor category loss to obtain the classification loss.
[0007] In one embodiment, the first weight is greater than the second weight.
[0008] In one embodiment, the large-category loss or the small-category loss is calculated based on a cross-entropy loss function.
[0009] In one embodiment, the method further includes: acquiring a target image to be identified; performing sub-category identification on the category of a second object in the target image by using a trained image recognition model to obtain a sub-category identification result of the second object.
[0010] In one embodiment, the method further includes: generating a vehicle driving strategy corresponding to the sub-category recognition result of the second object.
[0011] An embodiment of the present application discloses a training device for an image recognition model, comprising: a recognition module, used to perform sub-category recognition on a first object in a sample image through an image recognition model to be trained, and obtain a sub-category prediction result of the first object; a determination module, used to determine, based on the correspondence between sub-categories and major categories, a target major category to which the sub-category prediction result belongs, and obtain a major category prediction result; the correspondence includes N major categories, each major category includes M sub-categories, N is greater than or equal to 2, M is greater than or equal to 1, and the sub-categories in different major categories correspond to different vehicle driving strategies; an update module, used to calculate a classification loss based on the major category reference information of the first object and the major category prediction result, and to update the parameters in the image recognition model based on the classification loss.
[0012] An embodiment of the present application discloses an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor implements any one of the image recognition model training methods disclosed in the embodiment of the present application.
[0013] An embodiment of the present application discloses a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements a training method for any image recognition model disclosed in the embodiment of the present application.
[0014] Compared with the related art, the embodiments of the present application have the following beneficial effects:
[0015] The objects can be classified in advance according to the vehicle driving strategy to obtain the correspondence between the sub-categories and the major categories of the objects. The correspondence includes N major categories of object classification, each major category includes M sub-categories, N is greater than or equal to 2, M is greater than or equal to 1, and the sub-categories in different major categories correspond to different vehicle driving strategies.
[0016] The image recognition model to be trained can perform small-category recognition on the first object in the sample image to obtain a small-category prediction result of the first object; based on the correspondence between the small-category and the large-category, the large-category to which the small-category prediction result belongs is determined to obtain a large-category prediction result. The classification loss is calculated based on the large-category reference information of the first object and the large-category prediction result, so as to update the parameters in the image recognition model based on the classification loss.
[0017] The trained image recognition model can improve the accuracy of object sub-category recognition to a certain extent, and avoid identifying objects belonging to a certain category as another category. In addition, when the parameters of the image recognition model are reversely updated using the prediction results of the large category, the computing resources required for training can be reduced, and an image recognition model that can meet the needs of vehicle driving strategies can be trained under limited computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0019] Figure 1 is a method flow diagram of a method for training an image recognition model disclosed in an embodiment;
[0020] Figure 2 is a flowchart of a method for calculating classification loss disclosed in an embodiment;
[0021] Figure 3 is a flowchart of an image recognition method disclosed in an embodiment;
[0022] Figure 4 is a schematic diagram of the structure of a training device for an image recognition model disclosed in an embodiment;
[0023] Figure 5 The present invention is a schematic diagram of the structure of an electronic device disclosed in an embodiment. DETAILED DESCRIPTION
[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0025] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.
[0026] In related technologies, neural network models such as Convolutional Neural Network (CNN) are often used to identify object categories in images. Since different object categories may correspond to different vehicle driving strategies, when training a neural network model, downstream driving strategies are often referenced to guide the model's classification focus.
[0027] For example, when designing and training a neural network model, more attention is paid to the model's distinction between pedestrians and vehicles rather than the distinction between sedans and sport utility vehicles (SUVs) among vehicles.
[0028] Based on this, the following two training methods are often used in related technologies to train the model.
[0029] Method 1: Combine several categories of objects into the same category. For example, combine cars, SUVs, and buses into the category of vehicles, and combine adults and children into the category of pedestrians. Train the model so that it can identify the category to which the objects in the image belong.
[0030] Method 2: During training, different weights are set for the subcategories to which various objects belong in order to adjust the model's ability to recognize different subcategories. The trained model can accurately identify the subcategories to which objects in the image belong.
[0031] However, both of the above two methods have certain defects.
[0032] The model in method 1 is designed based on large-category recognition. When the driving strategy of downstream vehicles changes, the model needs to be modified accordingly, which makes it difficult to adapt to rapidly updated driving strategy changes.
[0033] Although the model in method 2 can identify small categories of objects, it requires more computing resources to train the model. If computing resources are limited, the training effect will be greatly reduced and it will not be able to meet the driving strategy requirements at all.
[0034] The embodiments of the present application disclose a method, device, electronic device and storage medium for training an image recognition model, which can train an image recognition model that can meet the requirements of vehicle driving strategies under the premise of limited computing resources.
[0035] See also Figure 1 , Figure 1 The following is a flow chart of a method for training an image recognition model disclosed in an embodiment. The method can be applied to electronic devices such as computers, vehicle terminals, and cloud servers, without specific limitation. Figure 1 As shown, the method may include the following steps:
[0036] 110. Perform sub-category recognition on the first object in the sample image by using the image recognition model to be trained to obtain a sub-category prediction result of the first object.
[0037] The image recognition model may include any neural network model, such as a CNN network, a feature pyramid network (FPN) network, etc., without specific limitation.
[0038] Before training the image recognition model, sample data for training may be acquired in advance. The sample data may include multiple frames of sample images, and each frame of sample image may include one or more objects to be recognized. For example, the sample data may include images of various types of objects such as SUVs, cars, adults, children, baby carriages, tricycles, etc.
[0039] Before training the image recognition model, different types of objects can be divided into N categories according to the downstream vehicle planning strategy, and each category can include M subcategories, so as to obtain the correspondence between subcategories and categories. N can be a positive integer greater than or equal to 2, M can be a positive integer greater than or equal to 1, and the subcategories in different categories correspond to different vehicle driving strategies.
[0040] Exemplarily, the vehicle driving strategy may include an obstacle avoidance strategy for moving obstacles, and the image recognition model needs to perform the task of identifying moving obstacles. It is expected that the image recognition model can distinguish between nine categories: cars, SUVs, buses, trucks, adults, children, tricycles, two-wheelers, and strollers. These nine categories can be further subdivided into smaller categories.
[0041] When avoiding moving obstacles, the vehicle driving strategy for controlling the vehicle to avoid a stroller may be similar to the vehicle driving strategy for controlling the vehicle to avoid an adult or a child, but the vehicle driving strategy for controlling the vehicle to avoid a stroller may be completely different from the vehicle driving strategy for controlling the vehicle to avoid a sedan or an SUV.
[0042] Therefore, the severity of misidentifying a baby stroller as an adult or a child is lower than the severity of misidentifying a baby stroller as a car or an SUV. In other words, misidentifying a baby stroller as an adult or a child is an error that can be tolerated to a certain extent, while misidentifying a baby stroller as a car or an SUV is relatively unacceptable.
[0043] Based on this, the above 9 subcategories can be divided into 3 major categories, namely vehicles, pedestrians, and non-motor vehicles. Among them, the major category of vehicles can include 4 subcategories: sedans, SUVs, buses, and trucks; the major category of pedestrians can include 3 subcategories: adults, children, and baby strollers; the major category of non-motor vehicles can include 2 subcategories: tricycles and two-wheelers.
[0044] In addition, in the embodiment of the present application, although the correspondence between the major category and the minor category is preset, the image recognition model performs minor category recognition on the first object in the sample image, and the image recognition model outputs the minor category prediction result of the first object.
[0045] 120. Based on the correspondence between the small category and the large category, determine the target large category to which the small category prediction result belongs, and obtain the large category prediction result.
[0046] After executing step 110 to obtain the sub-category prediction result output by the image recognition model, the electronic device can determine the target major category to which the sub-category prediction result belongs based on the preset correspondence between the sub-category and the major category.
[0047] Exemplarily, based on the correspondence between the sub-class and the major class in the aforementioned example, if the sub-class prediction result is a truck, the major class prediction result may be a vehicle; if the sub-class prediction result is a two-wheeled vehicle, the major class prediction result may be a non-motor vehicle.
[0048] 130. Calculate the classification loss according to the major category reference information and the major category prediction result of the first object.
[0049] 140. Update parameters in the image recognition model based on the classification loss.
[0050] The sample data used to train the image recognition model may include not only the sample image but also the general category reference information of the first object in the sample image. The general category reference information may be the correct general category classification category (Ground Truth) of the first object, which may be manually annotated.
[0051] The training of the image recognition model can be supervised learning, and the training objectives of the image recognition model can include: training the image recognition model to be able to predict the subcategory to which the object belongs, and expecting that after the subcategory predicted by the model is mapped into a major category based on a preset correspondence, the predicted major category classification is the same as the actual major category classification of the object.
[0052] Therefore, after the electronic device executes step 120 to map the small category prediction result predicted by the image recognition model for the first object to the large category prediction result, it calculates the classification loss between the large category prediction result and the actual large category classification information of the first object. The classification loss can be used to characterize the difference between the large category prediction result and the large category reference information.
[0053] Exemplarily, the electronic device may substitute the major category reference information and the major category prediction results into a preset loss function to calculate the classification loss. Exemplarily, the loss function may include a cross entropy loss function (CELoss), an L1 regularization loss function, a focal loss, an L2 regularization loss function, etc., without specific limitation.
[0054] The electronic device may update the parameters in the image recognition model based on methods such as gradient updating to minimize the classification loss; or, the number of training iterations exceeds a preset threshold, which is not specifically limited.
[0055] The electronic device uses an image recognition model to perform sub-category identification on the first object in the sample image, and then maps the sub-category prediction result into a major category prediction result, and uses the major category prediction result and major category reference information to update the parameters in the image recognition model, so that the trained image recognition model can distinguish objects belonging to different major categories, reducing the occurrence of errors in identifying objects belonging to one major category as another major category.
[0056] That is to say, the image recognition model obtained by training based on the training method disclosed in the aforementioned embodiment can improve the accuracy of identifying small categories of objects to a certain extent, and avoid identifying objects belonging to a certain large category as another large category.
[0057] At the same time, even if there is an error between the object subclass predicted by the image recognition model and the actual subclass, the impact on the vehicle driving strategy is small and such an error is tolerable. Therefore, when the parameters of the image recognition model are reversely updated using the large-class prediction results, the computing resources required for training can be reduced, thereby training an image recognition model that can meet the needs of vehicle driving strategies under limited resources.
[0058] For example, after a period of training, the image recognition model can accurately distinguish between adults and cars, but may confuse adults and children. If the image recognition model is required to accurately distinguish between adults and children, it needs to be trained further. However, at this point, the image recognition model can already meet the needs of the driving strategy, and training can be stopped, thereby saving computing resources.
[0059] In addition, the image recognition model disclosed in the embodiment of the present application is designed based on subcategory recognition. Even if the vehicle driving strategies corresponding to different subcategories change, since the image recognition model outputs the results of subcategory recognition, the changes to the image recognition model can be reduced.
[0060] It should be noted that Figure 1 The steps shown in FIG. 1 illustrate an iterative training process with a single frame sample image as input. The electronic device may perform the following steps on each frame sample image in the multiple frames sample images included in the sample data: Figure 1 The steps shown are repeated until the training end condition is met, such as the classification loss is minimized or the number of iterations exceeds a threshold, etc. When the training end condition is met, the electronic device can freeze the parameters in the image recognition model to obtain a trained image recognition model.
[0061] In one embodiment, after the electronic device executes the aforementioned step 120 and obtains the major category prediction result of the first object, if the major category corresponding to the major category prediction result includes at least two minor categories, the electronic device can calculate the classification loss by referring to the following steps.
[0062] See also Figure 2 , Figure 2 is a flowchart of a method for calculating classification loss disclosed in an embodiment, Figure 2 The steps shown may be an implementation of the aforementioned step 130. Figure 2 As shown, the following steps are included:
[0063] 210. Calculate the major category loss according to the major category reference information and the major category prediction result of the first object.
[0064] The major category losses can be calculated based on the preset loss function, which will not be elaborated in the following content.
[0065] 220. Calculate the sub-category loss according to the sub-category reference information and the sub-category prediction result of the first object.
[0066] In addition to the sample image and the major category reference information of the first object, the sample data may also include the minor category reference information of the first object. The minor category reference information may be the correct minor category classification category (ground truth) of the first object, which may be manually annotated.
[0067] The electronic device may also calculate the sub-category loss based on a preset loss function in combination with the sub-category reference information and the sub-category prediction result. The loss function used when calculating the major category loss and the minor category loss may be the same or different, and is not specifically limited.
[0068] Exemplarily, both the major category loss and the minor category loss can be calculated based on CE Loss.
[0069] It should be noted that there is no logical sequence relationship between step 210 and step 220, and step 210 and step 220 may also be executed simultaneously.
[0070] 230. Calculate classified losses based on major category losses and minor category losses.
[0071] Optionally, the electronic device may directly add the major category loss and the minor category loss, and use the sum as the classification loss.
[0072] Optionally, the electronic device can also adjust the weights of the major category loss and the minor category loss to meet the accuracy requirements of major category classification and minor category classification in different scenarios. That is, the electronic device can perform weighted sum calculation based on the first weight corresponding to the major category and the major category loss, and the second weight corresponding to the minor category and the minor category loss to obtain the classification loss.
[0073] Among them, the first weight corresponding to the major category is different from the second weight corresponding to the minor category, but the first weights corresponding to different major categories can be the same or different, and the second weights corresponding to different minor categories can be the same or different, and there is no specific limitation.
[0074] For example, the classification loss can be calculated by referring to the following formula:
[0075] Loss=Loss_inter*weight_inter+Loss_intra*weight_intra; Formula (1)
[0076] Among them, Loss can represent classification loss, Loss_inter can represent major category loss, weight_inter can represent major category weight corresponding to major category, Loss_intra can represent minor category loss, and weight_intra can represent the second weight corresponding to minor category.
[0077] Further optionally, the first weight may be set to be greater than the second weight, so as to focus the training on accurately distinguishing objects of different major categories while taking into account the accuracy of small category recognition.
[0078] Exemplarily, the first weight may be set to 10, and the second weight may be set to 1.
[0079] In the aforementioned embodiment, when a certain major category includes multiple minor categories, the electronic device can calculate the overall classification loss by combining the major category loss and the minor category loss, so that the trained image recognition model can improve the recognition accuracy of minor categories of objects while accurately identifying objects belonging to different major categories. In addition, by setting different weights, the focus of training can be adjusted.
[0080] In one embodiment, after the electronic device executes the aforementioned step 120 and obtains the major category prediction result of the first object, if the major category corresponding to the major category prediction result includes a subcategory, the electronic device can refer to the aforementioned step 210 to calculate the major category loss as the overall classification loss to reduce the waste of computing resources.
[0081] For example, it is assumed that objects are divided into 6 subcategories according to actual needs, namely c1, c2, c3, c4, c5, and c6. Among them, the vehicle driving strategies corresponding to c1, c2, and c3 are similar, and they are divided into the major category C1; the vehicle driving strategies corresponding to c4 and c5 are similar, but the differences from the vehicle driving strategies corresponding to c1, c2, and c3 are large, and they are divided into the major category C2; the vehicle driving strategy corresponding to c6 is different from the vehicle driving strategies of any of the aforementioned subcategories, and it is divided into the major category C3.
[0082] For example, each small category and large category can be represented separately in the form of one-hot label.
[0083] The one-hot encoding of a subclass can be expressed as [c1,c2,c3,c4,c5,c6]. For each subclass, only one value in the one-hot encoding is 1, and the rest are 0.
[0084] For example, the one-hot encoding of small class c1 can be expressed as: c1 = [1, 0, 0, 0, 0, 0];
[0085] The one-hot encoding of small class c2 can be expressed as: c2 = [0, 1, 0, 0, 0, 0];
[0086] The one-hot encoding of subclass c3 can be expressed as: c3 = [0, 0, 1, 0, 0, 0, 0].
[0087] The one-hot encoding of the major categories can be expressed as [C1, C2, C3]. In the one-hot encoding corresponding to each major category, only one value is 1, and the rest are 0. Among them: C1=c1+c2+c3, C2=c4+c5, C3=c6.
[0088] Based on the description of the unique hot encoding corresponding to the small category and the large category in the above content, the calculation of the classification loss can refer to the above formula (1).
[0089] Among them, the major category loss Loss_inter can be calculated by referring to the following formula:
[0090] Loss_inter=CE_loss([gt_C1,gt_C2,gt_C3],[pred_C1,pred_C2,pred_C3]); Formula (2)
[0091] Among them, CE_loss(·) can represent the cross entropy loss function, [gt_C1,gt_C2,gt_C3] can be the one-hot encoding of the major category reference information, and [pred_C1,pred_C2,pred_C3] can be the one-hot encoding of the major category prediction results.
[0092] Since the large category C3 only contains a small category c6, there is no need to distinguish between small categories. Therefore, it is only necessary to calculate the small category loss for the small categories in the large category C1 or the large category C2. Refer to the following formula for calculation:
[0093] Loss_intra_C1=CE_loss([gt_c1,gt_c2,gt_c3],[pred_c1,pred_c2,pred_c3]); Formula (3);
[0094] Among them, Loss_intra_C1 can represent the small class loss of the small class in the large class C1; [gt_c1, gt_c2, gt_c3] can be the one-hot encoding of the small class reference information, and [pred_c1, pred_c2, pred_c3] can be the one-hot encoding of the small class prediction result.
[0095] Loss_intra_C2=CE_loss([gt_c4,gt_c5],[pred_c4,pred_c5]); Formula (4);
[0096] Among them, Loss_intra_C2 can represent the small class loss of the small class in the large class C2; [gt_c4, gt_c5] can be the one-hot encoding of the small class reference information, and [pred_c4, pred_c5] can be the one-hot encoding of the small class prediction result.
[0097] The above embodiments disclose a method for training an image recognition model, and the trained model can be used to recognize the category of an object in an image. The following content introduces how to apply the trained image recognition model.
[0098] See also Figure 3 , Figure 3 is a flowchart of an image recognition method disclosed in an embodiment, the method includes steps that can be performed after the training steps disclosed in the above embodiment, Figure 3 The method shown can be executed by any of the aforementioned electronic devices. For ease of description, the following content is described using a vehicle-mounted terminal as an example. Figure 3 As shown, the method may include the following steps:
[0099] 310. Obtain a target image to be identified.
[0100] The vehicle may be provided with a camera device for capturing the environment around the vehicle, and the target image may be captured by the vehicle's camera device and transmitted to the vehicle-mounted terminal. The target image may be any frame of image to be recognized, and the second object included in the target image may be obstacles that the vehicle needs to avoid, such as pedestrians, motor vehicles, and non-motor vehicles, but is not limited thereto.
[0101] 320. Perform sub-category recognition on the category of the second object in the target image through the trained image recognition model to obtain a sub-category recognition result of the second object.
[0102] After being trained by the aforementioned training method, the image recognition model can extract the image features implicit in the target image, and perform subclass identification on the second object based on the image features, thereby obtaining the subclass identification result of the second object. In addition, there may be certain errors in the subclass identification of the second object by the image recognition model, but the error is tolerable. Even if the subclass identification result of the second object output by the image recognition model does not match the actual subclass classification of the second object, it is the subclass a1 belonging to the same large class that is mistakenly identified as subclass a2, rather than the subclass of the second object being mistakenly identified as subclass b1 belonging to another large class.
[0103] After obtaining the sub-category recognition result of the second object, the vehicle-mounted terminal can further process the sub-category recognition result of the second object based on different requirements. For example, output, store, and transmit the sub-category recognition result of the second object; or generate a corresponding control strategy according to the sub-category recognition result of the second object, without specific limitation.
[0104] In one embodiment, after executing step 320, the electronic device may further execute the following steps:
[0105] 330. Generate a vehicle driving strategy corresponding to the sub-category recognition result of the second object.
[0106] For example, assume that there are three different sub-categories of objects, namely cars, adults, and children. Adults and children belong to the same large category of pedestrians; cars belong to another large category of vehicles.
[0107] The vehicle driving strategy for a sedan may include: decelerating at a first deceleration; the vehicle driving strategy for an adult may include: decelerating at a second deceleration and honking; the vehicle driving strategy for a child may include: decelerating at a second deceleration and not honking. The first deceleration may be much smaller than the second deceleration.
[0108] That is to say, when the obstacles around the vehicle are pedestrians such as adults or children, the vehicle needs to slow down quickly to avoid collisions with pedestrians; when the obstacles around the vehicle are cars, the vehicle can slow down appropriately to reduce the possibility of collisions between cars, but considering that decelerating too quickly may injure the occupants, minor scratches are acceptable. In addition, considering that children are more easily frightened by whistles than pedestrians, the driving strategy for children does not include honking.
[0109] Assuming that the second object in the target image is actually an adult, there are three situations:
[0110] 1) If the sub-category recognition result of the second object output by the image recognition model is an adult, then the sub-category recognition is correct, and the vehicle driving strategy generated based on the sub-category recognition result is completely correct.
[0111] 2) If the output sub-category recognition result is a child, there is a certain error in the sub-category recognition, but the error between the vehicle driving strategy generated according to the sub-category recognition result and the completely correct vehicle driving strategy is not large. The deceleration restriction conditions are the same, and only the horn step is missing. However, it can also effectively avoid collisions between vehicles and pedestrians.
[0112] 3) If the output sub-category recognition result is a sedan, there will be a large error in the sub-category recognition, and the vehicle driving strategy generated based on the sub-category recognition result will also have a corresponding large error. The difference in deceleration limit conditions may cause the vehicle to collide with the adult in the target image.
[0113] However, based on the image recognition model disclosed in the embodiment of the present application, the vehicle driving strategy generated by the vehicle terminal after executing step 330 generally includes the aforementioned situation 1) and situation 2), and the probability of situation 3) occurring is extremely low and can be ignored.
[0114] It can be seen that in the image recognition method disclosed in the aforementioned embodiment, the sub-category recognition result of the second object output by the image recognition model has a certain accuracy, so that the vehicle driving strategy generated based on the sub-category recognition result can meet the actual driving needs.
[0115] It should be noted that the training method of the image recognition model and the image recognition method disclosed in the aforementioned embodiments can be executed by different electronic devices respectively; or, can also be executed by the same electronic device, without specific limitation.
[0116] For example, the image recognition model can be trained in electronic devices with strong computing power such as cloud servers and computers. The vehicle terminal can obtain the trained image recognition model from the cloud server or computer through Over-the-Air Technology (OTA) and other means, and store the trained image recognition model, so that the vehicle terminal can use the trained image recognition model to identify small categories of objects, and can save the computing resources of the vehicle terminal. In addition, the training effect of the image recognition model is also related to the computing power of the electronic device. Using electronic devices with strong computing power for training is also conducive to training an image recognition model that can better meet the needs.
[0117] See also Figure 4 , Figure 4 The structure diagram of a training device for an image recognition model disclosed in an embodiment is a schematic diagram, and the device can be applied to any electronic device disclosed in the above embodiments. Figure 4 As shown, the training device 400 for the image recognition model may include: a recognition module 410 , a determination module 420 , and an update module 430 .
[0118] The recognition module 410 is used to perform sub-category recognition on the first object in the sample image by using the image recognition model to be trained to obtain a sub-category prediction result of the first object;
[0119] A determination module 420 is used to determine the target major category to which the minor category prediction result belongs based on the correspondence between the minor category and the major category, and obtain the major category prediction result; the correspondence includes N major categories, each major category includes M minor categories, N is greater than or equal to 2, M is greater than or equal to 1, and the minor categories in different major categories correspond to different vehicle driving strategies;
[0120] The updating module 430 is used to calculate the classification loss according to the major category reference information and the major category prediction result of the first object, and update the parameters in the image recognition model according to the classification loss.
[0121] In one embodiment, the major category corresponding to the major category prediction result includes at least two minor categories; the updating module 430 may include: a calculation unit and an updating unit.
[0122] The calculation unit can be used to calculate the major category loss based on the major category reference information and major category prediction results of the first object; and calculate the minor category loss based on the minor category reference information and minor category prediction results of the first object; and calculate the classification loss based on the major category loss and the minor category loss.
[0123] The update unit can be used to update the parameters in the image recognition model according to the classification loss calculated by the calculation unit.
[0124] In one embodiment, the calculation unit may also be used to perform weighted sum calculation based on a first weight corresponding to the major category and the major category loss, and a second weight corresponding to the minor category and the minor category loss to obtain a classification loss.
[0125] In one embodiment, the first weight is greater than the second weight.
[0126] In one embodiment, the large-class loss or the small-class loss is calculated based on a cross-entropy loss function.
[0127] In one embodiment, the image recognition model training device 400 may further include: an acquisition module and a recognition module.
[0128] An acquisition module, which can be used to acquire a target image to be identified;
[0129] The recognition module can be used to perform sub-category recognition on the category of the second object in the target image through the trained image recognition model to obtain the sub-category recognition result of the second object.
[0130] In one embodiment, the image recognition model training device 400 may further include: a generation module.
[0131] The generation module can be used to generate a vehicle driving strategy corresponding to the sub-category recognition result of the second object.
[0132] The training device for implementing the image recognition model disclosed in the aforementioned embodiment can improve the accuracy of identifying small categories of objects to a certain extent, so as to avoid identifying objects belonging to a certain large category as another large category. In addition, when the parameters of the image recognition model are reversely updated using the large category prediction results, the computing resources required for training can be reduced, thereby training an image recognition model that can meet downstream driving strategy requirements under limited resources.
[0133] The embodiment of the present application discloses an image recognition device, which can be applied to any of the aforementioned electronic devices, and the device includes: an acquisition module and a recognition module. Optionally, the device may also include a generation module.
[0134] The operations performed by the acquisition module, the identification module and the generation module are similar to those in the aforementioned embodiment and will not be described in detail below.
[0135] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device disclosed in an embodiment. Figure 5 As shown, the electronic device 500 may include:
[0136] A memory 510 storing executable program codes;
[0137] a processor 520 coupled to the memory 510;
[0138] The processor 520 calls the executable program code stored in the memory 510 to execute any one of the image recognition model training methods disclosed in the embodiments of the present application.
[0139] It should be noted that Figure 5 The electronic device shown may also include a power supply, input buttons, a camera, a speaker, a screen, an RF circuit, a Wi-Fi module, a Bluetooth module, a sensor and other components not shown, which are not described in detail in this embodiment.
[0140] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute any one of the image recognition model training methods disclosed in the embodiment of the present application.
[0141] An embodiment of the present application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute any one of the image recognition model training methods disclosed in the embodiment of the present application.
[0142] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.
[0143] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0144] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0146] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.
[0147] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0148] The training method, device, electronic device and storage medium of the image recognition model disclosed in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the present application are described in detail using specific examples herein. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A training method for an image recognition model, It is characterized in that The method comprises: Performing sub-category recognition on a first object in a sample image by using an image recognition model to be trained, thereby obtaining a sub-category prediction result of the first object; the sample image carries major category reference information of the first object and minor category reference information of the first object; Based on the correspondence between the small category and the large category, determine the target large category to which the small category prediction result belongs, and obtain the large category prediction result; the correspondence includes N large categories of object classification, each large category includes M small categories, N is greater than or equal to 2, M is greater than or equal to 1, and the small categories in different large categories correspond to different vehicle driving strategies; the large category corresponding to the large category prediction result includes at least two small categories; Calculating a major category loss according to the major category reference information of the first object and the major category prediction result; Calculating a subclass loss based on the subclass reference information of the first object and the subclass prediction result; Performing weighted sum calculation according to a first weight corresponding to the major category and the major category loss, and a second weight corresponding to the minor category and the minor category loss, to obtain a classification loss, wherein the first weight is greater than the second weight; Parameters in the image recognition model are updated according to the classification loss.
2. The method according to claim 1, It is characterized in that The large-category loss or the small-category loss is calculated based on a cross-entropy loss function.
3. The method according to claim 1 or 2, It is characterized in that The method further comprises: Acquire a target image to be identified; The category of the second object in the target image is sub-classified by using the trained image recognition model to obtain a sub-class recognition result of the second object.
4. The method according to claim 3, It is characterized in that The method further comprises: A vehicle driving strategy corresponding to the sub-category recognition result of the second object is generated.
5. A training device for an image recognition model, It is characterized in that include: A recognition module, used for performing sub-category recognition on a first object in a sample image by using an image recognition model to be trained, and obtaining a sub-category prediction result of the first object; The sample image carries the major category reference information of the first object and the minor category reference information of the first object; A determination module, used to determine the target major category to which the minor category prediction result belongs based on the correspondence between the minor category and the major category, and obtain the major category prediction result; the correspondence includes N major categories, each major category includes M minor categories, N is greater than or equal to 2, M is greater than or equal to 1, and the minor categories in different major categories correspond to different vehicle driving strategies; the major category corresponding to the major category prediction result includes at least two minor categories; An updating module, configured to calculate a classification loss according to the major category reference information of the first object and the major category prediction result, and to update parameters in the image recognition model according to the classification loss; The updating module comprises a calculating unit and an updating unit; The calculation unit is used to calculate the major category loss according to the major category reference information of the first object and the major category prediction result; and, calculating a subclass loss according to the subclass reference information of the first object and the subclass prediction result; and performing weighted sum calculation according to a first weight corresponding to the major category and the major category loss, and a second weight corresponding to the minor category and the minor category loss, to obtain a classification loss, wherein the first weight is greater than the second weight; The updating unit is used to update the parameters in the image recognition model according to the classification loss calculated by the calculating unit.
6. An electronic device, It is characterized in that The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 4.
7. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Neural network training method, device and system, and storage medium
CN108875934A
Obstacle Avoidance Method and Apparatus for Autonomous Driving Vehicle
US20210001841A1
