Image recognition method and device, electronic equipment and medium

Abstract

The present disclosure provides an image recognition method and device, electronic equipment and medium, and relates to the technical field of automatic driving. The method comprises: obtaining image information collected by a vehicle during driving; identifying the image information through a target image recognition model to obtain an image recognition result; the target image recognition model is obtained in the following manner: when the image recognition model is used, if a leading head of the image recognition model outputs an incorrect result, an auxiliary head is added to the image recognition model, the auxiliary head learns sample data corresponding to the incorrect result, the auxiliary head migrates learned features to the leading head to obtain a trained image recognition model; and the auxiliary head in the trained image recognition model is deleted to obtain the target image recognition model. Compared with the original image recognition model, no new module is added to the target image recognition model, the structure remains simple, the model parameters are relatively few, and the image recognition speed is fast.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of automatic driving, and particularly relates to an image recognition method and device, electronic equipment and medium. BACKGROUND

[0002] As an important application of artificial intelligence, automatic driving technology has developed rapidly in recent years. As the most important part of automatic driving, the safety of automatic driving depends on the correctness and robustness of the perception system. In the perception system, models are usually used to identify vehicles, pedestrians, traffic lights, etc. When a bad case (wrong or undesirable result) occurs in the model, a new module can be added to the model to improve the bad case. However, adding a new module to the model increases the complexity of the model, and when the model is used to identify image information, the calculation speed of the model slows down, and the speed of image recognition correspondingly slows down. SUMMARY

[0003] To overcome the problems in the related art, the present disclosure provides an image recognition method, device, electronic equipment and medium.

[0004] According to a first aspect of an embodiment of the present disclosure, an image recognition method is provided, the method comprising: acquiring image information collected by a vehicle during driving; inputting the image information into a target image recognition model to obtain an image recognition result output by a dominant head in the target image recognition model, wherein the target image recognition model is obtained by deleting an auxiliary head from a trained image recognition model, the trained image recognition model comprising the auxiliary head and the dominant head, and the trained image recognition model is obtained by migrating features learned by the auxiliary head to the dominant head after learning sample data corresponding to an error result by the auxiliary head in the case of an error result output by the image recognition model.

[0005] Optionally, the trained image recognition model is obtained by: acquiring first basic sample data and sample data corresponding to the error result, wherein the first basic sample data is sample data for which the image recognition model does not output an error result; training the auxiliary head by the first basic sample data and the sample data corresponding to the error result, and training the dominant head by the first basic sample data; and migrating the features learned by the auxiliary head to the dominant head to obtain the trained image recognition model.

[0006] Optionally, the migrating the feature learned by the auxiliary head to the main head comprises: obtaining a first main loss function corresponding to the main head, obtaining a first auxiliary loss function corresponding to the auxiliary head, and obtaining a distillation loss function corresponding to the image recognition model; and migrating the feature learned by the auxiliary head to the main head by using the first main loss function, the first auxiliary loss function, and the distillation loss function.

[0007] Optionally, the first auxiliary loss function is a product of the first main loss function and a first weight coefficient.

[0008] Optionally, the method further comprises: obtaining second basic sample data; and training an initial training model by using the second basic sample data to obtain the image recognition model.

[0009] Optionally, the training the initial training model by using the second basic sample data to obtain the image recognition model comprises: training the initial training model by using the second basic sample data to obtain a trained initial training model; obtaining a second main loss function corresponding to a main head in a recognition layer and a second auxiliary loss function corresponding to an auxiliary head in the recognition layer for the trained initial training model; and updating the trained initial training model by using the second main loss function and the second auxiliary loss function to obtain the image recognition model.

[0010] Optionally, the second auxiliary loss function is a product of the second main loss function and a second weight coefficient.

[0011] Optionally, the target image recognition model is a multi-task model for automatic driving.

[0012] According to a second aspect of the embodiments of the present disclosure, an image recognition device is provided, which comprises: a collection module configured to obtain image information collected by a vehicle during driving; and an identification module configured to input the image information into a target image recognition model to obtain an image recognition result output by a main head in the target image recognition model, wherein the target image recognition model is obtained by deleting an auxiliary head from a trained image recognition model, the trained image recognition model comprises the auxiliary head and the main head, and the trained image recognition model is obtained by migrating a feature learned by the auxiliary head to the main head after learning sample data corresponding to an error result by using the auxiliary head in a case that the image recognition model outputs the error result.

[0013] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of the method of the first aspect when executing the instructions.

[0014] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions which, when executed by a processor, implement the steps of the method provided by the first aspect of the present disclosure.

[0015] The present disclosure provides an image recognition method and device, an electronic device and a medium. Image information collected by a vehicle during driving is obtained, and the image information is input into a target image recognition model to obtain an image recognition result output by a dominant head of the target image recognition model. The target image recognition model is obtained in the following manner: when the image recognition model is used, if the dominant head of the image recognition model outputs an error result, an auxiliary head is added to the image recognition model, the auxiliary head learns sample data corresponding to the error result, and then the auxiliary head migrates the learned features to the dominant head to obtain a trained image recognition model; and the auxiliary head in the trained image recognition model is deleted to obtain the target image recognition model. In the present disclosure, the image information is recognized by the target image recognition model, and the dominant head of the target image recognition model outputs the image recognition result. Compared with the original image recognition model used to recognize the image information, no new module is added to the target image recognition model. Therefore, the target image recognition model does not become complex due to the need to solve the bad case problem, and still maintains a simple structure, so that the model is easy to debug and deploy, and subsequent image recognition is facilitated. In addition, since the model structure is simple and the model parameters are relatively few, the speed of image recognition by the model is faster.

[0016] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] Figure 1 is a flowchart of an image recognition method according to an exemplary embodiment;

[0019] Figure 2 is a schematic diagram of a target image recognition model according to an exemplary embodiment;

[0020] Figure 3 is a schematic diagram of a target image recognition model according to an exemplary embodiment;

[0021] Figure 4 is a flowchart of another image recognition method according to an example embodiment;

[0022] Figure 5 is a schematic diagram of an image recognition model according to an example embodiment;

[0023] Figure 6 is a schematic diagram of an image recognition model with an auxiliary head added according to an example embodiment;

[0024] Figure 7 is a schematic diagram of training a main head and an auxiliary head according to an example embodiment;

[0025] Figure 8 is a flowchart of another image recognition method according to an example embodiment;

[0026] Figure 9 is a block diagram of an image recognition device according to an example embodiment;

[0027] Figure 10 is a functional block diagram of an electronic device according to an example embodiment;

[0028] Figure 11 is a block diagram of a server according to an example embodiment. DETAILED DESCRIPTION

[0029] The example embodiments will be described in detail herein with reference to the attached drawings. In the following description, same numbers refer to same elements in all figures. The following detailed description includes specific details for the purpose of providing a thorough understanding of the example embodiments. However, it will be apparent to those skilled in the art that the example embodiments can be practiced without these specific details. In some instances, well-known structures and components are not described in detail in order to avoid obscuring the example embodiments.

[0030] As an important application of artificial intelligence, autonomous driving technology has made great progress in recent years. As the most important part of autonomous driving, the safety of autonomous driving depends on the correctness of the perception system. In the perception system, models are usually used to identify vehicles, pedestrians, traffic lights, etc. When a bad case occurs in the model, a new module can be added to the model to improve the situation. A bad case refers to a situation where the model makes errors or performs poorly in prediction. However, adding a new module to the model increases the complexity of the model. When using the model to identify image information, the model's calculation speed slows down due to the increased complexity of the model, which slows down the speed of image recognition. Moreover, as the complexity of the model increases, it becomes more difficult to debug and deploy the model.

[0031] To solve the above problems, the present disclosure provides an image recognition method, device, electronic equipment and medium. In the case that the leading head of the model outputs an incorrect result, an auxiliary head is added to the model, the sample data corresponding to the incorrect result is learned by the auxiliary head, and then the auxiliary head migrates the learned features to the leading head. After successful migration, the auxiliary head in the model is deleted, and the model is used to identify image information. The model does not add new modules, so the model structure is simple, the model debugging and deployment are difficult, the subsequent image recognition is facilitated, and the model parameters are relatively small due to the simple model structure, so the image recognition speed by the model is faster. The image recognition method is described in subsequent embodiments.

[0032] Please refer to Figure 1 , Figure 1 is a flowchart of an image recognition method according to an exemplary embodiment, which can be applied to the image recognition device 200, electronic equipment and computer readable storage medium shown in Figure 9 The present embodiment takes the application to the electronic equipment as an example, wherein the electronic equipment can be a vehicle central processor, processing equipment, etc. on the vehicle, and the electronic equipment can also be a server in communication connection with the vehicle. The following will be described in detail with respect to the flow shown in Figure 1 The image recognition method can specifically include the following steps:

[0033] Step S110, acquiring image information collected by the vehicle during driving.

[0034] The perception system of the vehicle includes a camera, which is installed on the vehicle and captures image information during driving of the vehicle.

[0035] Optionally, in order to obtain a wider view angle during driving of the vehicle, the number of cameras on the vehicle can be multiple, and the multiple cameras are arranged at different positions of the vehicle, and the cameras at different positions collect image information of different view angles.

[0036] Step S120, inputting the image information into a target image recognition model to obtain an image recognition result output by a leading head in the target image recognition model, wherein the target image recognition model is obtained by deleting an auxiliary head in a trained image recognition model, the trained image recognition model includes the auxiliary head and the leading head, and the trained image recognition model is obtained by migrating features learned by the auxiliary head to the leading head after learning sample data corresponding to an incorrect result by the auxiliary head in the case that the image recognition model outputs the incorrect result.

[0037] The image information is input into the target image recognition model, and the image is recognized by the target image recognition model. The image recognition result output by the leading head in the target image recognition model.

[0038] In an embodiment, referring to Figure 2 , the target image recognition model 110 is a single-task model for automatic driving. The target image recognition model 110 includes a backbone, a neck, and a leading head. The backbone is used to extract features. The neck is a connection layer connecting the backbone and the leading head, which compresses and reduces the dimension of the features extracted by the backbone, reduces the number of parameters and the computational complexity, and improves the recognition speed of the subsequent leading head. The image information is input into the target image recognition model 110, the features of the image information are extracted by the backbone in the target image recognition model 110, the extracted image features are sent to the neck by the target image recognition model 110, the image features are compressed and reduced in dimension by the neck, and then the compressed and reduced image features are input into the leading head, the image features are recognized by the leading head, and the image recognition result is output. For the single-task model shown in Figure 2 , the model recognizes one task, and the image recognition result includes the recognition result of one type of object. For example, the recognition task is to recognize vehicles, and the target image recognition model is a single-task model for recognizing vehicles. Therefore, the image recognition result includes the recognition result of the photographed vehicle. Alternatively, if the image information photographed includes multiple vehicles, the image recognition result includes the recognition result of the multiple vehicles.

[0039] In another embodiment, referring to Figure 3 , the target image recognition model 110 can be a multi-task model for automatic driving. The target image recognition model 110 includes a backbone, a neck, and multiple leading heads. Each leading head in the multiple leading heads corresponds to one recognition task. The image information is input into the target image recognition model 110, the features of the image information are extracted by the backbone in the target image recognition model 110, the extracted image features are sent to the neck by the target image recognition model 110, the compressed and reduced image features are input into the multiple leading heads, respectively, the image features are recognized by each leading head, and the image recognition result is output by each leading head. For the multi-task model shown in Figure 3The multi-task model shown in the image recognition result contains the recognition results of multiple categories of objects. For example, the target image recognition model is a multi-task model for recognizing vehicles, zebra crossings, traffic lights, and pedestrians. The image recognition result can include the recognition results of the photographed vehicles, zebra crossings, traffic lights, and pedestrians. It should be noted that the image recognition result does not necessarily contain the recognition results of all types of objects. The specific type of recognition result included in the image recognition result is determined according to the actual image information obtained.

[0040] The image recognition method provided in this embodiment obtains image information collected during vehicle driving, and then inputs the image information into a target image recognition model to obtain an image recognition result output by a dominant head of the target image recognition model. The target image recognition model is obtained in the following manner: when the dominant head of the image recognition model outputs an incorrect result during use of the image recognition model, an auxiliary head is added to the image recognition model, the auxiliary head learns sample data corresponding to the incorrect result, and then the auxiliary head migrates the learned features to the dominant head to obtain a trained image recognition model; and the auxiliary head in the trained image recognition model is deleted to obtain the target image recognition model. In this embodiment, the target image recognition model is used to recognize image information, and the dominant head of the target image recognition model outputs an image recognition result. Compared with the original image recognition model used to recognize image information, no new module is added to the target image recognition model. Therefore, the target image recognition model does not become complex due to the need to solve the bad case problem, and still maintains a simple structure, so that the model is easy to debug and deploy, and subsequent image recognition is facilitated. In addition, because the model structure is simple and the model parameters are relatively few, the speed of image recognition by the model is faster.

[0041] Optionally, the image information collected during vehicle driving can be stored in a sample database to expand the training samples. When the model needs to be trained subsequently, the image information can be used as training samples.

[0042] Referring to Figure 4 , the trained image recognition model is obtained in the following manner:

[0043] In step S210, first basic sample data and sample data corresponding to the incorrect result are obtained. The first basic sample data is sample data for which the image recognition model does not output an incorrect result.

[0044] In one case, the image recognition model is deployed in an electronic device and has been used for image recognition. Referring to Figure 5, the image recognition model 120 includes a backbone network, a bottleneck layer, and a leading head. When image recognition is performed on the image recognition model 120 using certain image data, and the leading head of the image recognition model 120 outputs an incorrect result, the image recognition model 120 needs to be adjusted. Then, the aforementioned image data is taken as sample data corresponding to the incorrect result. And the first basic sample data is obtained, wherein the first basic sample data is sample data for which the image recognition model does not output an incorrect result. It can be understood that the first basic sample data can be sample data for which the image recognition model does not output an incorrect result, or the first sample data can not have been input to the image recognition model and not have been recognized by the image recognition model.

[0045] It should be noted that the image recognition model is not limited to Figure 5 a single-task model, but can also be a multi-task model. When the image recognition model is a multi-task model, if any leading head of the multi-task model outputs an incorrect result, the model needs to be adjusted.

[0046] In this case, the image recognition model is deployed in an electronic device and has been used for image recognition. Image data is collected during the driving of a vehicle, and the image data is input to the image recognition model. If the leading head of the image data outputs an incorrect result, the image data is taken as sample data corresponding to the incorrect result. And the first sample data is obtained from the sample database. Optionally, the first sample data in the sample database can be data that is collected during the driving of the same vehicle, is recognized by the image recognition model, and does not output an incorrect result.

[0047] In another case, the image recognition model is trained by an initial training model and has not been deployed in an electronic device. Please refer to Figure 6 , the image recognition model 120 includes a backbone network, a bottleneck layer, an auxiliary head, and a leading head. When image recognition is performed on the image recognition model 120 using certain image data, and the leading head of the image recognition model 120 outputs an incorrect result, the image recognition model 120 needs to be adjusted. Then, the aforementioned image data is taken as sample data corresponding to the incorrect result. And the first basic sample data is obtained.

[0048] In this case, the image recognition model is trained by an initial training model and has not been deployed in an electronic device. The sample database stores a plurality of sample data, and the sample data are all image data. When any sample data in the database is input to the image recognition model, if the leading head of the image recognition model outputs an incorrect result, the sample data is determined to be sample data corresponding to the incorrect result. The remaining sample data in the sample database, except for the sample data corresponding to the incorrect result, can be taken as first sample data.

[0049] Step S220, training the auxiliary head by the first basic sample data and the sample data corresponding to the error result, and training the leading head by the first basic sample data.

[0050] In this embodiment, the auxiliary head is used to learn the characteristics of the sample data. Please refer to Figure 6 For an image recognition model deployed in an electronic device as shown in Figure 5 , an auxiliary head is added to the image recognition model. For an image recognition model not deployed in an electronic device, the model has an auxiliary head by itself, as shown in Figure 6 . Optionally, the auxiliary head can be set in the middle network layer where the leading head is located.

[0051] For ease of description, please refer to Figure 7 , the first basic sample data is sample data d1, and the sample data corresponding to the error result is sample data d2. The sample data d1 and the sample data d2 are input into the image feature model, and the sample data d1 and the sample data d2 pass through the backbone network and the bottleneck layer in turn, and then the sample data d1 and the sample data d2 are input into the auxiliary head. The auxiliary head is trained by the sample data d1 and the sample data d2. And the sample data d1 is input into the leading head, and the auxiliary head is trained by the sample data d1. Figure 7

[0052] It should be noted that the image recognition model is not limited to the single-task model shown in Figure 7 , but can also be a multi-task model. Optionally, when the image recognition model is a multi-task model, an auxiliary head can be set for each leading head, and the auxiliary head corresponding to each leading head processes the bad case corresponding to the respective task. Optionally, when the image recognition model is a multi-task model, an auxiliary head can also be set only for the leading head that appears a bad case, so as to reduce the number of auxiliary heads in the model, thereby reducing the parameters of the model and improving the subsequent training speed.

[0053] Step S230, migrating the characteristics learned by the auxiliary head to the leading head to obtain the trained image recognition model.

[0054] ​In an embodiment, a first main loss function corresponding to the main head, a first auxiliary loss function corresponding to the auxiliary head, and a distillation loss function corresponding to the image recognition model are obtained. Then, the features learned by the auxiliary head are migrated to the main head through the first main loss function, the first auxiliary loss function, and the distillation loss function. In this way, the features learned by the auxiliary head are migrated to the main head by making the feature representation of the main head closer to the feature representation of the auxiliary head through the first main loss function, the first auxiliary loss function, and the distillation loss function. For example, the sum of the first main loss function, the first auxiliary loss function, and the distillation loss function is calculated to obtain a first total loss function, and the features learned by the auxiliary head are migrated to the main head through the first total loss function.

[0055] Optionally, the first auxiliary loss function is a product of the first main loss function and a first weight coefficient. The first weight coefficient is set according to training requirements and can be 0.1, 0.15, etc.

[0056] In the embodiment, the main head is trained by the first basic sample data, so that the model maintains the previous function through the main head, and the auxiliary head is trained by the first basic sample data and the sample data corresponding to the error result, so that the model can solve the bad case problem through the auxiliary head. Then, the features learned by the auxiliary head are migrated to the main head to obtain a trained image recognition model. The main head in the trained image recognition model can maintain the previous function and solve the bad case problem.

[0057] Optionally, after obtaining the trained image recognition model, the auxiliary head in the trained image recognition model is deleted to obtain a target image recognition model. Deleting the auxiliary head can reduce the parameters in the target image recognition model to improve the image recognition speed of the target image recognition model.

[0058] Optionally, when the number of sample data used for training is small, different sample data can be spliced. For example, the first basic sample data and the sample data corresponding to the error result in the sample database are spliced to obtain new sample data, so as to expand the number of samples and increase the richness of the samples in the sample database.

[0059] Optionally, the method further includes obtaining second basic sample data, wherein the second basic sample data is any sample data of the sample database. The initial training model is trained by the second basic sample data to obtain the image recognition model. Please refer to Figure 8 The image recognition model is trained in the following manner:

[0060] Step S310, training the initial training model through the second basic sample data to obtain a trained initial training model.

[0061] The second basic sample data is input into the initial training model, the initial training model is trained, and a trained initial training model is obtained.

[0062] Optionally, the initial training model comprises a backbone network, a bottleneck layer, an auxiliary head and a leading head, the second basic sample data is input into the leading head and the auxiliary head through the backbone network and the bottleneck layer in sequence, the leading head and the auxiliary head are trained respectively, and the trained initial training model is obtained.

[0063] Step S320, for the trained initial training model, a second leading loss function corresponding to the leading head in the identification layer and a second auxiliary loss function corresponding to the auxiliary head in the identification layer are obtained.

[0064] The second auxiliary loss function is a product of the second leading loss function and a second weight coefficient. Optionally, the second weight coefficient is set according to training requirements and can be 0.1, 0.15, etc.

[0065] Optionally, the second weight coefficient can be the same as the first weight coefficient.

[0066] Step S330, updating the trained initial training model through the second leading loss function and the second auxiliary loss function to obtain the image recognition model.

[0067] Illustratively, the sum of the second leading loss function and the second auxiliary loss function is calculated to obtain a second total loss function, and the parameters of the trained initial training model are updated through the second total loss function to obtain the image recognition model. Illustratively, the intermediate network layer is updated through the second total loss function, which can be understood as updating the auxiliary head and the leading head.

[0068] Based on the same inventive concept, the present disclosure provides an image recognition device 200, please refer to Figure 9 , the image recognition device 200 comprises:

[0069] The acquisition module 210 is configured to acquire image information collected by the vehicle during driving;

[0070] The identification module 220 is configured to input the image information into a target image identification model to obtain an image identification result output by a dominant head in the target image identification model, wherein the target image identification model is obtained by deleting an auxiliary head from a trained image identification model, the trained image identification model includes the auxiliary head and the dominant head, and the trained image identification model is obtained by migrating features learned by the auxiliary head to the dominant head after the auxiliary head learns sample data corresponding to an error result in a case where the image identification model outputs the error result.

[0071] Optionally, the image identification apparatus 200 further includes:

[0072] The first data acquisition module is configured to acquire first basic sample data and sample data corresponding to the error result, wherein the first basic sample data is sample data for which the image identification model does not output an error result.

[0073] The first training module is configured to train the auxiliary head by using the first basic sample data and the sample data corresponding to the error result, and train the dominant head by using the first basic sample data.

[0074] The migration module is configured to migrate the features learned by the auxiliary head to the dominant head to obtain the trained image identification model.

[0075] Optionally, the migration module includes:

[0076] The first loss function acquisition module is configured to acquire a first dominant loss function corresponding to the dominant head, acquire a first auxiliary loss function corresponding to the auxiliary head, and acquire a distillation loss function corresponding to the image identification model.

[0077] The feature migration module is configured to migrate the features learned by the auxiliary head to the dominant head by using the first dominant loss function, the first auxiliary loss function, and the distillation loss function.

[0078] Optionally, the first auxiliary loss function is a product of the first dominant loss function and a first weight coefficient.

[0079] Optionally, the image identification apparatus 200 further includes:

[0080] The second data acquisition module is configured to acquire second basic sample data.

[0081] The second training module is configured to train an initial training model by using the second basic sample data to obtain the image identification model.

[0082] Optionally, the second training module includes:

[0083] an initial training module configured to train an initial training model by using the second basic sample data, and obtain a trained initial training model;

[0084] a second loss function obtaining module configured to obtain, for the trained initial training model, a second dominant loss function corresponding to a dominant head in the recognition layer, and a second auxiliary loss function corresponding to an auxiliary head in the recognition layer;

[0085] an updating module configured to update the trained initial training model by using the second dominant loss function and the second auxiliary loss function, and obtain the image recognition model.

[0086] Optionally, the second auxiliary loss function is a product of the second dominant loss function and a second weight coefficient.

[0087] Optionally, the target image recognition model is a multi-task model for automatic driving.

[0088] As to the image recognition apparatus 200 in the above embodiments, the specific manners in which the modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0089] The present disclosure also provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of the image recognition method provided by the present disclosure.

[0090] Figure 10 is a block diagram of an electronic device 600 according to an example embodiment. For example, the electronic device 600 can be a device on a vehicle, such as a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or another type of vehicle. The vehicle can be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0091] Referring to Figure 10 , the electronic device 600 can include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. The electronic device 600 can include more or fewer subsystems, and each subsystem can include multiple components. In addition, the electronic device 600 can be interconnected between each subsystem and each component by wired or wireless means.

[0092] In some embodiments, the infotainment system 610 can include a communication system, an entertainment system, a navigation system, and the like.

[0093] The perception system 620 can include several types of sensors for sensing information of the environment surrounding the electronic device 600. For example, the perception system 620 can include a global positioning system (which can be a GPS system, a Beidou system, or other positioning system), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0094] The decision control system 630 can include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0095] The drive system 640 can include components that provide powered motion for the electronic device 600. In one embodiment, the drive system 640 can include an engine, an energy source, a transmission system, and wheels. The engine can be one or a combination of an internal combustion engine, an electric motor, an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0096] Some or all functions of the electronic device 600 are controlled by the computing platform 650. The computing platform 650 can include at least one processor 651 and a memory 652, and the processor 651 can execute instructions 653 stored in the memory 652.

[0097] The processor 651 can be any conventional processor, such as commercially available CPUs. The processor can also include a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.

[0098] The memory 652 can be implemented by any type of volatile or nonvolatile memory devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic or optical disk.

[0099] In addition to the instructions 653, the memory 652 can also store data, such as road maps, route information, the position, direction, speed, and the like of the vehicle. The data stored in the memory 652 can be used by the computing platform 650.

[0100] In the embodiments of the present disclosure, the processor 651 can execute the instructions 653 to complete all or part of the steps of the image recognition method described above.

[0101] Figure 11 A server is shown, which is connected with a vehicle and receives image information captured by the vehicle during driving. Referring to Figure 11 The server 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, for storing instructions, such as application programs, executable by the processing component 1922. The application programs stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the method described above.

[0102] The server 1900 can also include a power supply component 1926 configured to perform power management of the server 1900, a wired or wireless network interface 1950 configured to connect the server 1900 to a network, and an input / output interface 1958. The server 1900 can operate based on an operating system stored in the memory 1932.

[0103] In another exemplary embodiment, a computer program product is also provided, which contains a computer program executable by a programmable device, the computer program having code portions for performing the image recognition method described above when executed by the programmable device.

[0104] To sum up, the image recognition method, device, electronic equipment and medium provided by the present disclosure obtain image information collected by a vehicle during driving, input the image information into a target image recognition model, obtain an image recognition result output by a dominant head of the target image recognition model, and the target image recognition model is obtained in the following way: when the image recognition model is used, if the dominant head of the image recognition model outputs an error result, an auxiliary head is added to the image recognition model, the auxiliary head learns sample data corresponding to the error result, and then the auxiliary head migrates the learned features to the dominant head to obtain a trained image recognition model; the auxiliary head in the trained image recognition model is deleted to obtain the target image recognition model. In the present disclosure, the target image recognition model is used to recognize image information, and the dominant head of the target image recognition model outputs an image recognition result. Compared with the original image recognition model used to recognize image information, no new module is added to the target image recognition model. Therefore, the target image recognition model does not become complex due to the need to solve the bad case problem, and still maintains a simple structure, so that the model is easy to debug and deploy, and subsequent image recognition is facilitated. In addition, since the model structure is simple and the model parameters are relatively few, the speed of image recognition by the model is faster.

[0105] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

[0106] It is to be understood that the disclosure is not limited to the precise construction herein described and as shown in the attached drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the disclosure. The scope of the disclosure is limited only by the claims that follow.

Claims

1. An image recognition method characterized by, The method comprises: acquiring image information collected by a vehicle during driving; inputting the image information into a target image recognition model to obtain an image recognition result output by a dominant head in the target image recognition model, wherein the target image recognition model is obtained by deleting an auxiliary head from a trained image recognition model, the trained image recognition model comprises the auxiliary head and the dominant head, and the trained image identification model is obtained by migrating features learned by the auxiliary head to the dominant head after learning sample data corresponding to an error result by the auxiliary head in a case where the image recognition model outputs the error result; The trained image recognition model is obtained by the following method: acquiring first basic sample data and sample data corresponding to the error result, wherein the first basic sample data is sample data for which the image recognition model does not output an error result; training the auxiliary head by using the first basic sample data and the sample data corresponding to the error result, and training the dominant head by using the first basic sample data; migrating the features learned by the auxiliary head to the dominant head to obtain the trained image recognition model.

2. The method of claim 1, wherein, The method comprises: acquiring a first dominant loss function corresponding to the dominant head, a first auxiliary loss function corresponding to the auxiliary head, and a distillation loss function corresponding to the image recognition model; migrating the features learned by the auxiliary head to the dominant head by using the first dominant loss function, the first auxiliary loss function, and the distillation loss function.

3. The method of claim 2, wherein, The first auxiliary loss function is a product of the first dominant loss function and a first weight coefficient.

4. The method of any one of claims 1-2, wherein, Further comprising: acquiring second basic sample data; training an initial training model by using the second basic sample data to obtain the image recognition model.

5. The method of claim 4, wherein, The method comprises: training an initial training model by using the second basic sample data to obtain a trained initial training model; for the trained initial training model, acquiring a second dominant loss function corresponding to a dominant head in a recognition layer and a second auxiliary loss function corresponding to an auxiliary head in the recognition layer; updating the trained initial training model by using the second dominant loss function and the second auxiliary loss function to obtain the image recognition model.

6. The method of claim 5, wherein, The second auxiliary loss function is a product of the second dominant loss function and a second weight coefficient.

7. The method of any one of claims 1-2, wherein, The target image recognition model is a multi-task model for automatic driving.

8. An image recognition apparatus characterized by comprising: The device comprises: an acquisition module configured to acquire image information collected by a vehicle during driving; The identification module is configured to input the image information into a target image identification model, and obtain an image identification result output by a dominant head in the target image identification model, wherein the target image identification model is obtained by deleting an auxiliary head from a trained image identification model, the trained image identification model comprises the auxiliary head and the dominant head, and the trained image identification model is obtained by migrating features learned by the auxiliary head to the dominant head after learning sample data corresponding to an error result by the auxiliary head in a case that the image identification model outputs the error result. The trained image identification model is obtained by the following method: obtaining first basic sample data and sample data corresponding to the error result, wherein the first basic sample data is sample data for which the image identification model does not output an error result; training the auxiliary head by using the first basic sample data and the sample data corresponding to the error result, and training the dominant head by using the first basic sample data; migrating the features learned by the auxiliary head to the dominant head to obtain the trained image identification model.

9. An electronic device, comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of the method of any one of claims 1-7 when executing the instructions.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The program instructions, when executed by the processor, implement the steps of the method of any one of claims 1-7.

Images