Image recognition method and device, neural network model training method and device

By introducing hypernetwork modules and subnetwork modules into neural network models, using predictors to automatically select appropriate subnets for image recognition, and adjust parameters during training, the problem that the model cannot adapt to data changes in the existing technology is solved, and the improvement of the improvement of the problem that the model cannot adapt to data changes in the existing technology. Identification effect and efficiency.

CN112784912BActive Publication Date: 2024-11-26BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110127552.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-29
Publication Date
2024-11-26
Estimated Expiration
2041-01-29

AI Technical Summary

Technical Problem

The existing neural network models cannot be adjusted according to the actual input data during data processing, resulting in insufficient processing efficiency and accuracy, and cannot adapt to the needs of different images to be identified.

Method used

A multi-cascaded neural network model is adopted, including hypernetwork modules and subnetwork modules, and the appropriate subnetwork is determined based on the input feature map through the predictor for identification and processing, and the parameters are adjusted during the training process to take into account both accuracy and efficiency.

Benefits of technology

The adaptability and recognition effect of neural network models to different images to be identified is improved, and the performance and efficiency of data processing are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112784912B_ABST
    Figure CN112784912B_ABST
Patent Text Reader

Abstract

The present disclosure provides an image recognition method and device, a neural network model training method and device, and relates to the field of artificial intelligence, specifically the field of computer vision and deep learning. The implementation scheme is: input the image to be recognized into the neural network model; and perform recognition of the image to be recognized based on at least part of the network modules close to the input side of the neural network model in multiple network modules, wherein, for any super network module in at least one super network module and its corresponding predictor, when the super network module receives the feature map output by the previous network module, the feature map is input into the predictor corresponding to the super network module to obtain the predicted value output by the predictor, based on the predicted value, determine one of the at least two sub-networks of the super network module as the selected sub-network, and in the super network module, use the selected sub-network to perform recognition of the image to be recognized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the field of computer vision and deep learning, and specifically to a training method and device for a neural network model, an image recognition method and device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides an image recognition method and apparatus, a neural network model training method and apparatus, a computer device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of the present disclosure, there is provided an image recognition method using a neural network model, wherein the neural network model comprises a plurality of cascaded network modules, the plurality of cascaded network modules comprise at least one super network module, each of the at least one super network module comprises at least two sub-networks, the neural network model further comprises a predictor corresponding to each of the at least one super network module, and the recognition method comprises: inputting an image to be recognized into the neural network model; and performing recognition of the image to be recognized based on at least part of the network modules in the plurality of network modules close to the input side of the neural network model, wherein, for any super network module in at least one super network module and its corresponding predictor, when the super network module receives a feature map output by a previous network module, the feature map is input into the predictor corresponding to the super network module to obtain a prediction value output by the predictor, based on the prediction value, determining one of the at least two sub-networks of the super network module as a selected sub-network, and in the super network module, using the selected sub-network to perform recognition of the image to be recognized.

[0006] According to another aspect of the present disclosure, a training method for a neural network model is provided, wherein the neural network model includes a plurality of cascaded network modules, the plurality of cascaded network modules include at least one super network module, and the training method includes: obtaining a plurality of first training sets, wherein each of the plurality of first training sets includes a plurality of first training samples and their true classifications; determining the first training set corresponding to each of the plurality of cascaded network modules according to a first preset mode; and adjusting the parameters of the network module using the first training set corresponding to each of the plurality of network modules, wherein, for each super network module in at least one super network module and its corresponding first training set, in response to each of the plurality of first training samples in the first training set being input into the neural network model, obtaining the predicted classification and the first calculation duration of the first training sample output by the super network module, and adjusting the parameters of the super network module based on the true classification, the predicted classification and the first calculation duration of each of the plurality of first training samples in the first training set.

[0007] According to another aspect of the present disclosure, there is provided an image recognition device, comprising: a neural network model, comprising a plurality of cascaded network modules, the plurality of cascaded network modules comprising at least one super network module, each of the at least one super network module comprising at least two sub-networks, and the neural network model further comprising a predictor corresponding to each of the at least one super network module; an input unit, configured to input an image to be recognized into the neural network model; the neural network model is configured to perform recognition of the image to be recognized based on at least part of the network modules in the plurality of network modules close to the input side of the neural network model, wherein, for any super network module in at least one super network module and its corresponding predictor, when the super network module receives a feature map output by a previous network module, the feature map is input into the predictor corresponding to the super network module to obtain a prediction value output by the predictor, based on the prediction value, one of the at least two sub-networks of the super network module is determined to be a selected sub-network, and in the super network module, the selected sub-network is used to perform recognition of the image to be recognized.

[0008] According to another aspect of the present disclosure, a training device for a neural network model is provided, wherein the neural network model includes a plurality of cascaded network modules, the plurality of cascaded network modules include at least one super network module, and the training device includes: a first acquisition unit, configured to acquire a plurality of first training sets, wherein each of the plurality of first training sets includes a plurality of first training samples and their true classifications; a first determination unit, configured to determine, according to a first preset mode, a first training set corresponding to each of the plurality of cascaded network modules; and a first adjustment unit, configured to adjust parameters of the network module using the first training set corresponding to each of the plurality of network modules, wherein, for each super network module in at least one super network module and its corresponding first training set, in response to each of the plurality of first training samples in the first training set being input into the neural network model, the predicted classification and first calculation duration of the first training sample output by the super network module are acquired, and the parameters of the super network module are adjusted based on the true classification, predicted classification and first calculation duration of each of the plurality of first training samples in the first training set.

[0009] According to another aspect of the present disclosure, a computer device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor is configured to execute the computer program to implement the steps of the above method.

[0010] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program implements the steps of the above method when executed by a processor.

[0011] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the steps of the above method are implemented when the computer program is executed by a processor.

[0012] According to one or more embodiments of the present disclosure, a neural network model including a super network module obtained by training according to the present disclosure can take into account the requirements for data processing accuracy and processing efficiency, thereby improving the data processing performance of the neural network model.

[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0015] Figure 1 A schematic diagram showing an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure;

[0016] Figure 2a A flowchart of an image recognition method using a neural network model according to an embodiment of the present disclosure is shown;

[0017] Figure 2b A flowchart of an image recognition method of a hypernetwork module and its corresponding predictor according to an embodiment of the present disclosure is shown;

[0018] Figure 3 A schematic diagram of the structure of a super network according to an embodiment of the present disclosure is shown;

[0019] Figure 4a A flowchart of a method for training a neural network model according to an embodiment of the present disclosure is shown;

[0020] Figure 4b A method for adjusting parameters of a hypernetwork module and its corresponding predictor according to an embodiment of the present disclosure is shown;

[0021] Figure 5 A structural block diagram of an image recognition device according to an embodiment of the present disclosure is shown;

[0022] Figure 6 A structural block diagram of a training device for a neural network model according to an embodiment of the present disclosure is shown;

[0023] Figure 7 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0025] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0026] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0027] As one aspect of the application of artificial intelligence technology, computers can simulate human thinking processes based on trained neural network models and identify input data.

[0028] In the related art, once the neural network model is trained, it will be applied to the data processing process with fixed parameters. This results in the model being unable to be adjusted specifically according to the actual input data during the data processing application process.

[0029] Based on this, the present disclosure proposes an image recognition method and device, a training method and device for a neural network model, a computer device, a computer-readable storage medium, and a computer program product. The neural network model including a super network module obtained by training using the present disclosure can take into account the requirements for data processing accuracy and processing efficiency, thereby improving the data processing performance of the neural network model. When the neural network model obtained by training based on the training method in the present disclosure is applied to the field of image recognition, it is possible to quickly search for a sub-network in the super network module that meets the requirements for accuracy and processing efficiency based on the input image to be recognized for targeted recognition processing, thereby improving the adaptability of the neural network model to different images to be recognized and improving the overall recognition effect.

[0030] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0031] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0032] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of a training method for a neural network model or an image recognition method.

[0033] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0034] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0035] The user may use client devices 101, 102, 103, 104, 105 and / or 106 to obtain the image to be recognized and the result of image recognition. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0036] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing a variety of different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0037] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0038] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0039] The computing unit in the server 120 may run one or more operating systems including any of the above-mentioned operating systems and any commercially available server operating system. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc. The server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of the client devices 101, 102, 103, 104, 105, and 106. In some embodiments, the server 120 may include one or more applications, for example, applications for services such as target detection and recognition, signal conversion, etc. based on data such as images, videos, voices, texts, digital signals, etc., to process deep learning task requests such as voice interaction, text classification, image recognition, or key point detection received from the client devices 101, 102, 103, 104, 105, and 106, and accept various media data as training sample data for deep learning tasks, such as image data, audio data, or text data. The server can also train the neural network model including the super network module using the training samples according to the specific deep learning task, test each sub-network in the super network module, and determine the structure and parameters of the neural network model used to perform the above deep learning task according to the test results of each sub-network. After the training of the neural network model is completed, the server 120 can also automatically search for an appropriate network structure through the model structure to perform the corresponding task.

[0040] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0041] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The data repository 130 may reside in various locations. For example, the data repository used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The data repository 130 may be of different types. In some embodiments, the data repository used by the server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to a command.

[0042] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0043] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0044] Figure 2a and 2b The present invention illustrates an image recognition method using a neural network model according to an exemplary embodiment of the present disclosure, wherein the neural network model includes a plurality of cascaded network modules, the plurality of cascaded network modules include at least one super network module, each of the at least one super network module includes at least two sub-networks, and the neural network model further includes a predictor corresponding to each of the at least one super network module, such as Figure 2a As shown, the recognition method may include: step S201, inputting the image to be recognized into the neural network model; and step S202, performing recognition of the image to be recognized based on at least part of the network modules in the multiple network modules close to the input side of the neural network model, wherein, for any super network module in at least one super network module and its corresponding predictor, as Figure 2bAs shown, step S202-1, when the super network module receives the feature map output by the previous network module, the feature map is input into the predictor corresponding to the super network module to obtain the prediction value output by the predictor, step S202-2, based on the prediction value, one of the at least two sub-networks of the super network module is determined as the selected sub-network, and step S202-3, in the super network module, the selected sub-network is used to perform recognition of the image to be recognized. Thus, by performing image recognition through the neural network model including the super network module, a preferred sub-network in the super network module can be automatically searched for image recognition according to the input image to be recognized, thereby improving the adaptability of the neural network model to different images to be recognized and enhancing the recognition effect.

[0045] At the same time, when using the neural network model to perform deep learning tasks, such as image recognition tasks, the hardware device can select a network structure that matches the current input data to perform processing based on the current input data. In this way, in the process of processing a large amount of different input data, it can effectively save the computing resources, memory resources, and video memory resources of the hardware device, improve the computing power and processing efficiency of the hardware device in the process of performing deep learning tasks, and reduce the configuration requirements of deep learning tasks for the hardware device.

[0046] According to some embodiments, the neural network model may be a multi-layer neural network model, wherein the network module may be a specific number of consecutive layers. For example, the neural network model may be a 10-layer neural network model, wherein the first 5 layers constitute a network module in the neural network model, wherein the last 5 layers constitute another network module in the neural network model, which is not limited here.

[0047] According to some embodiments, a super network may be a network structure including at least two sub-networks at the same time. The super network may include multiple layers of search space, each layer of search space includes multiple sub-structures. Each sub-structure may serve as a layer structure in a sub-network. Figure 3 FIG. 1 shows a schematic diagram of the structure of a hypernetwork of an exemplary embodiment. Figure 3 As shown, the super network 300 includes three layers of search space, and each layer of search space includes three substructures.

[0048] It can be understood that the search space in the hypernetwork is not limited to three layers, and the number of substructures in each layer of the search space is not limited to three. The number of substructures in different search spaces can be the same or different. Figure 3 The number of substructures in each layer shown in FIG. 1 is only an example and is not limiting on the present disclosure.

[0049] A substructure is determined from each layer of the search space of the super network, and after being connected in sequence according to the layer order of the search space, a subnetwork of the super network can be formed.

[0050] In one embodiment, for example, Figure 3 Taking the super network 300 as an example, a substructure 312 is selected from the first layer search space, a substructure 323 is selected from the second layer search space, and a substructure 331 is selected from the third layer search space, and the selected substructures are sequentially connected to obtain a subnetwork. In another embodiment, different subnetworks can also be obtained by changing the number of layers.

[0051] After completing step S201 to input the image to be recognized into the neural network model, step S202 can be executed to perform recognition of the image to be recognized based on at least some network modules among the multiple network modules that are close to the input side of the neural network model.

[0052] According to some embodiments, performing recognition of an image to be recognized based on at least some network modules in a plurality of network modules close to the input side of a neural network model may include: for any network module in a plurality of network modules, in response to the network module receiving the image to be recognized or one of the feature maps output by a previous network module, the network module outputs a predicted classification of the image to be recognized and its confidence; in response to the confidence being greater than a preset threshold, determining the predicted classification as a recognition result. Thus, it may be determined whether to terminate the image recognition process based on the confidence output by the network module.

[0053] Specifically, for any network module among multiple cascaded network modules, when the confidence of the network module output is greater than a preset threshold, the predicted classification output by the network module is determined as the recognition result, and the subsequent network modules of the neural network model no longer participate in the recognition processing of the image to be recognized.

[0054] For example, when the recognition difficulty level of the input image to be recognized is low, the ideal confidence level can be obtained only through the partial network modules close to the input end of the neural network model, thereby quickly obtaining the ideal recognition result without losing the calculation accuracy. Conversely, when the recognition difficulty level of the input image to be recognized is high, the entire network module of the neural network model can be used for calculation to improve the recognition accuracy.

[0055] According to some embodiments, for any network module in the multiple cascaded network modules that is not located at the output end of the neural network model, in response to the confidence level being less than a preset threshold, the feature map output by the network module can be input to the next network module. In this way, when the confidence level is not ideal, the subsequent network module can be used to perform further calculations on the image to be recognized, thereby improving the recognition accuracy.

[0056] According to some embodiments, for a network module located at the output end of a neural network model, in response to a confidence level being less than a preset threshold, it may be determined that the prediction classification is a recognition result.

[0057] Based on the above, the neural network model can adjust the number of network modules involved in the recognition processing based on the input image to be recognized, thereby improving the adaptability of the neural network model to different images to be recognized, optimizing the recognition effect, and improving the processing speed on the hardware device without losing recognition accuracy.

[0058] For any super-network module in at least one super-network module and its corresponding predictor, step S202-1, when the super-network module receives the feature map output by the previous network module, the feature map is input into the predictor corresponding to the super-network module to obtain the prediction value output by the predictor, step S202-2, based on the prediction value, determine that one of the at least two sub-networks of the super-network module is a selected sub-network, and step S202-3, in the super-network module, use the selected sub-network to perform recognition of the image to be recognized.

[0059] The hypernetwork module includes at least two subnetworks. Different subnetworks have different processing results and processing speeds for the same image due to differences in parameters, structures and other factors. Based on the above, the neural network model can select the subnetwork with the best effect from the hypernetwork module for the image to be recognized through the predictor to process the image to be recognized, thereby improving the adaptability of the neural network model to different images to be recognized and optimizing the recognition effect.

[0060] Among them, the parameters of the predictor can be adjusted according to the requirements of accuracy and speed for image recognition in actual processing, so that the trained predictor can determine the optimal sub-network that meets the accuracy and speed requirements according to the input image to be recognized.

[0061] Figure 4a and 4b is a flow chart showing a training method of a neural network model according to an exemplary embodiment of the present disclosure, wherein the neural network model includes a plurality of cascaded network modules, and the plurality of cascaded network modules include at least one super network module, such as Figure 4a As shown, the training method may include: step S401, obtaining multiple first training sets, wherein each of the multiple first training sets includes multiple first training samples and their true classifications; step S402, determining the first training set corresponding to each of the multiple cascaded network modules according to the first preset mode; and step S403, using the first training set corresponding to each of the multiple network modules to adjust the parameters of the network module, wherein for each super network module in at least one super network module and its corresponding first training set, as shown in FIG. Figure 4bAs shown, step S403-1, in response to each of the multiple first training samples in the first training set being input into the neural network model, obtaining the predicted classification and the first calculation duration of the first training sample output by the super network module, and step S403-2, based on the real classification, predicted classification and the first calculation duration of each of the multiple first training samples in the first training set, adjusting the parameters of the super network module. Thus, in the process of training the neural network model, it is possible to pay attention to both the final output result and the intermediate output result. In the training of the super network module, the parameters of the super network module are adjusted based on the real classification, predicted classification and the first calculation duration, so that the trained super network module has good performance in both speed and accuracy.

[0062] The neural network model trained in the above method can be applied to perform corresponding deep learning tasks and process corresponding media data, such as image recognition tasks, target detection tasks based on media data such as images, etc.

[0063] With respect to step S402, according to some embodiments, the plurality of first training sets have at least two different recognition difficulty levels, and according to the first preset mode, determining the first training set corresponding to each of the plurality of cascaded network modules may include: determining the first training set corresponding to each of the plurality of network modules according to the position of the network module in the neural network model, wherein, for any two of the plurality of network modules, the recognition difficulty level of the training set corresponding to the network module close to the input side of the neural network model is not greater than the recognition difficulty level of the training set corresponding to the other network module. Thus, in the process of training the neural network model, different first training sets with different recognition difficulty levels may be used to conduct targeted training on network modules located at different positions.

[0064] The first training set with a specific difficulty level includes first training samples corresponding to the recognition difficulty level of the first training set. The higher the recognition difficulty level of the first training sample, the less likely it is to be recognized.

[0065] According to some embodiments, the first training sample may be a sample image.

[0066] For example, when the sample image only includes a complete cat, the recognition difficulty level of the sample image is low (i.e., it is easy to recognize that there is a cat in the sample image); when the sample image includes other interference objects, or only includes a part of a cat, the recognition difficulty level of the sample image is high (i.e., it is difficult to recognize that there is a cat in the sample image).

[0067] According to some embodiments, the method for obtaining at least two first training sets with different recognition difficulty levels may include obtaining multiple first training samples; in response to inputting each of the multiple first training samples into a trained grading model, the grading model outputs a predicted classification of the first training sample and its confidence; using at least one of the predicted classification and confidence of each of the multiple first training samples, determining the recognition difficulty level of the first training sample; and constructing a first training set with a corresponding recognition difficulty level based on at least one first training sample with the same recognition difficulty level. Thus, first training sets with different recognition difficulty levels can be quickly obtained.

[0068] In one embodiment, each of the plurality of first training samples may have a corresponding true classification, and thus the recognition difficulty level of the first training sample may be determined according to the consistency between the predicted classification and the true classification of each of the plurality of first training samples. Specifically, when the predicted classification and the true classification are consistent, the recognition difficulty level of the first training sample is determined to be low; when the predicted classification and the true classification are inconsistent, the recognition difficulty level of the first training sample is determined to be high.

[0069] In another embodiment, the recognition difficulty level of each first training sample among the plurality of first training samples may be determined according to the confidence level of the first training sample. Specifically, the confidence level of the first training sample is negatively correlated with the recognition difficulty level of the first training sample.

[0070] In another embodiment, the recognition difficulty level of the first training sample may be determined based on the consistency between the predicted classification and the actual classification of the first training sample and the confidence of the first training sample.

[0071] According to some embodiments, the recognition difficulty level of each first training sample in the plurality of first training samples may also be determined by manual labeling, thereby constructing a required first training set with a corresponding recognition difficulty level.

[0072] For any two network modules among the multiple network modules, the recognition difficulty level of the first training set corresponding to the network module close to the input side of the neural network model is not greater than the recognition difficulty level of the first training set corresponding to the other network module. That is, the recognition difficulty level of the first training set corresponding to the network module closer to the input side of the neural network model is smaller, and the recognition difficulty level of the first training set corresponding to the network module farther from the input side of the neural network model is greater.

[0073] With respect to step S403, according to some embodiments, adjusting the parameters of each network module using the first training set corresponding to each network module in the plurality of network modules may include: for any two network modules in the plurality of network modules, first adjusting the parameters of the network modules close to the input side of the neural network model. Thus, subsequent network modules can be trained based on the trained network modules close to the input side of the neural network model, thereby improving the training effect of subsequent network modules.

[0074] Specifically, the first network module located at the input end of the neural network model can be trained first. The first training set corresponding to the first network module includes multiple first training samples and their true classifications. Each of the multiple first training samples in the first training set is input into the neural network model to obtain the predicted classification of the first training sample output by the first network module. Based on the true classification and predicted classification of each of the multiple first training samples, the parameters of the first network module are adjusted. In this way, the training of the first network module in the neural network model can be achieved. Since the first network module is trained with the first training set with a lower recognition difficulty level, the first network module trained in this way can more easily extract feature information from the image to be recognized with a lower recognition difficulty, thereby improving the recognition effect.

[0075] After the parameters of the first network module are determined, the next network module (i.e., the second network module) cascaded with the first network module in the neural network model is trained. The first training set corresponding to the second network module includes multiple first training samples and their true classifications. Each of the multiple first training samples in the first training set is input into the neural network model to obtain the predicted classification of the first training sample output by the second network module. Based on the true classification and predicted classification of each of the multiple first training samples, the parameters of the second network module are adjusted. In the process of adjusting the parameters of the second network module, although the first network module also participates in the calculation of the first training sample, the parameters of the first network module no longer change. Thus, the training of the second network module in the neural network model can be realized. Since the second network module can be trained with a first training set with a higher recognition difficulty level (compared with the first training set corresponding to the first network module), the second network module trained in this way can more easily extract feature information from the image to be recognized with a higher recognition difficulty level, thereby improving the recognition effect.

[0076] After the parameters of the second network module are determined, the next network module (i.e., the third network module) cascaded with the second network module in the neural network model is trained, and so on, the training of each network module in the neural network model can be completed in sequence.

[0077] For step S403-1 and step S403-2, according to some embodiments, the first calculation duration is the time taken by the GPU to process the training task of each first training sample. In response to a first training sample being input into the neural network model, the first calculation duration is the GPU processing duration from the first training sample being input into the neural network model to the predicted classification of the first training sample output by the super network module.

[0078] Since the super network can contain at least two sub-networks at the same time, and the first calculation durations corresponding to different sub-networks can be different. Therefore, in the process of training the super network module, the parameters of the super network module are adjusted based on the real classification, predicted classification and first calculation duration of each of the multiple first training samples in the first training set, and the calculation accuracy (calculation accuracy is determined according to the real classification and predicted classification of the first training sample) and calculation speed (calculation speed is determined according to the first calculation duration) of each sub-network in the super network module can be comprehensively considered to adjust the parameters of the super network module until the super network module converges.

[0079] According to some embodiments, each of the at least one super network module includes at least two sub-networks, and the neural network model also includes a predictor corresponding to each of the at least one super network module. The training method may also include: for each of the at least one super network module, after adjusting the parameters of the super network module, obtaining a second training set; using the second training set to adjust the parameters of the predictor, so that the predictor can determine that one of the at least two sub-networks of the super network module is the selected sub-network according to the feature map output by the previous network module of the super network module. Thus, the predictor obtained by training can select the optimal sub-network in the super network module for the input image to be recognized in the process of image recognition to process the image to be recognized, so as to achieve targeted processing of different images to be recognized, effectively improve the adaptability of the neural network model to different images to be recognized, and improve the recognition effect.

[0080] According to some embodiments, the second training set includes multiple second training samples, and adjusting the parameters of the predictor corresponding to the super network module using the second training set may include the following steps: in response to each of the multiple second training samples in the second training set being input into the neural network model, obtaining a feature map of the second training sample output by the previous network module of the super network module; inputting the feature map of the second training sample into the predictor corresponding to the super network module to obtain a predicted sub-network corresponding to the second training sample output by the predictor; inputting the feature map of the second training sample into the super network module to determine the real sub-network corresponding to the second training sample; and adjusting the parameters of the predictor corresponding to the network module based on the real sub-network and predicted sub-network of each of the multiple second training samples.

[0081] According to some embodiments, the second training sample may be a sample image.

[0082] According to some embodiments, the second training set also includes a true classification of each second training sample in a plurality of second training samples, and inputting a feature map of the second training sample into the super network module to determine a true subnetwork of the second training sample may include: in response to inputting a feature map of the second training sample into the super network module, obtaining a predicted classification of the second training sample output by each of at least two subnetworks of the super network module and a second calculation time corresponding to the subnetwork, and determining, based on the true classification, predicted classification and second calculation time of each second training sample in the plurality of second training samples, one of the at least two subnetworks of the super network module as a true subnetwork.

[0083] In the process of training the predictor corresponding to the hypernetwork module, based on the real classification, predicted classification and second calculation duration of each second training sample in multiple second training samples, one of the at least two subnetworks of the hypernetwork module is determined to be a real subnetwork, so that the trained predictor can predict the most preferred subnetwork in the hypernetwork module based on the image to be recognized input into the neural network model.

[0084] According to some embodiments, wherein each of at least one super network module includes at least two sub-networks, the training method may further include: for each of the at least one super network module, after adjusting the parameters of the super network module, obtaining a test set; and using the test set, determining that one of the at least two sub-networks of the super network module is a selected sub-network.

[0085] According to some embodiments, the test set may include multiple test samples and their true classifications; for each super network module in at least one super network module, in response to each test sample in the test set being input into a neural network model, the predicted classification of the test sample output by each of the at least two sub-networks of the super network module and the third calculation duration corresponding to the sub-network are obtained, and based on the true classification, predicted classification and third calculation duration of each test sample in the multiple test samples, one of the at least two sub-networks of the super network module is determined as the selected sub-network. Thus, the optimal sub-network can be determined from the super network module according to the requirements for recognition accuracy and recognition speed.

[0086] According to some embodiments, the test sample may be a test image.

[0087] According to some embodiments, the test set may be adaptively selected based on the actual application scenario. For example, the recognition difficulty level of the test samples included in the test set, or the ratio of the number of test samples with different recognition difficulty levels in the test set may be determined based on the actual application scenario.

[0088] According to another aspect of the present disclosure, Figure 5 As shown, an image recognition device 500 is also provided, including: a neural network model 510, including a plurality of cascaded network modules 511-515, the plurality of cascaded network modules 511-515 including at least one super network module 512-513, each of the at least one super network module 512-513 including at least two sub-networks, the neural network model 510 also includes a predictor 522-523 corresponding to each of the at least one super network module 512-513; an input unit 520, configured to input an image to be recognized into the neural network model 510; the neural network model 510 is configured to be based at least on Some of the network modules in the multiple network modules 511-515 that are close to the input side of the neural network model 510 perform recognition of the image to be recognized, wherein, for any super network module in the at least one super network module 512-513 and its corresponding predictor, when the super network module receives a feature map output by the previous network module, the feature map is input into the predictor corresponding to the super network module to obtain a prediction value output by the predictor, and based on the prediction value, one of the at least two sub-networks of the super network module is determined to be a selected sub-network, and in the super network module, the selected sub-network is used to perform recognition of the image to be recognized.

[0089] It can be understood that the number of network modules and / or super network modules in the above neural network model is only an illustrative example, and the present disclosure does not limit the number of network modules and / or super network modules in the neural network model.

[0090] According to some embodiments, the neural network model is further configured to output a predicted classification of an image to be identified and its confidence level for any one of a plurality of network modules in response to the network module receiving an image to be identified or one of the feature maps output by a previous network module; the recognition device further includes: a determination unit configured to determine the predicted classification as a recognition result in response to a confidence level being greater than a preset threshold.

[0091] According to some embodiments, the determination unit is further configured to: for any network module among the multiple cascaded network modules that is not located at the output end of the neural network model, in response to a confidence level being less than a preset threshold, input a feature map output by the network module to the next network module.

[0092] According to another aspect of the present disclosure, Figure 6 As shown, a training device 600 for a neural network model is also disclosed, wherein the neural network model includes multiple cascaded network modules, and the multiple cascaded network modules include at least one super network module. The training device 600 includes: a first acquisition unit 601, configured to acquire multiple first training sets, wherein each of the multiple first training sets includes multiple first training samples and their true classifications; a first determination unit 602, configured to determine the first training set corresponding to each of the multiple cascaded network modules according to a first preset mode; and a first adjustment unit 603, configured to use the first training set corresponding to each of the multiple network modules to adjust the parameters of the network module, wherein, for each super network module in at least one super network module and its corresponding first training set, in response to each of the multiple first training samples in the first training set being input into the neural network model, the predicted classification and first calculation duration of the first training sample output by the super network module are obtained, and based on the true classification, predicted classification and first calculation duration of each of the multiple first training samples in the first training set, the parameters of the super network module are adjusted.

[0093] According to some embodiments, multiple first training sets have at least two different recognition difficulty levels, and the determination unit is further configured to determine the first training set corresponding to each network module among the multiple network modules based on the position of the network module in the neural network model, wherein, for any two network modules among the multiple network modules, the recognition difficulty level of the training set corresponding to the network module close to the input side of the neural network model is not greater than the recognition difficulty level of the training set corresponding to the other network module.

[0094] According to some embodiments, each of at least one super network module includes at least two sub-networks, the neural network model also includes a predictor corresponding to each of the at least one super network module, and the training device also includes a second acquisition unit and a second adjustment unit, wherein, for each of the at least one super network module, after adjusting the parameters of the super network module, the second acquisition unit is configured to acquire a second training set; the second adjustment unit is configured to use the second training set to adjust the parameters of the predictor corresponding to the super network module, so that the predictor can determine that one of the at least two sub-networks of the super network module is the selected sub-network based on the feature map output by the previous network module of the super network module.

[0095] According to some embodiments, each of at least one super network module includes at least two sub-networks, and the training device also includes a third acquisition unit and a second determination unit, wherein, for each of the at least one super network module, after adjusting the parameters of the super network module, the third acquisition unit is configured to acquire a test set; and the second determination unit is configured to use the test set to determine that one of the at least two sub-networks of the super network module is a selected sub-network.

[0096] According to some embodiments, the first adjustment unit is further configured to: for any two network modules among the multiple network modules, first adjust the parameters of the network modules close to the input side of the neural network model.

[0097] According to another aspect of the present disclosure, a computer device is further provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor is configured to execute the computer program to implement the steps of the above method.

[0098] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is further provided, on which a computer program is stored, wherein the computer program implements the steps of the above method when executed by a processor.

[0099] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein the steps of the above method are implemented when the computer program is executed by a processor.

[0100] According to an embodiment of the present disclosure, an electronic device, a readable storage medium and a computer program product are also provided.

[0101] refer to Figure 7 , a block diagram of an electronic device 700 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0102] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0103] A plurality of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 may be any type of device capable of inputting information to the device 700, the input unit 706 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device, and may include but is not limited to a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and / or a remote controller. The output unit 707 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 708 may include but is not limited to a disk, an optical disk. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a 1302.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0104] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a training method for a neural network model or an image recognition method. For example, in some embodiments, the training method for a neural network model or an image recognition method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the training method for the neural network model or the image recognition method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute a training method of a neural network model or an image recognition method in any other appropriate manner (eg, by means of firmware).

[0105] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0107] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0109] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0110] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0111] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0112] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. An image recognition method using a neural network model, wherein: The neural network model includes a plurality of cascaded network modules, the plurality of cascaded network modules include at least one super network module, each of the at least one super network module includes at least two sub-networks, and the neural network model also includes a predictor corresponding to each of the at least one super network module. The identification method comprises: Inputting the image to be recognized into the neural network model; and The image to be recognized is recognized at least based on some network modules of the plurality of network modules close to the input side of the neural network model, wherein: For any hypernetwork module in the at least one hypernetwork module and its corresponding predictor, When the super network module receives the feature map output by the previous network module, the feature map is input into the predictor corresponding to the super network module to obtain the predicted value output by the predictor. Based on the predicted value, determining one of the at least two sub-networks of the super-network module as a selected sub-network, and In the super network module, the selected sub-network is used to perform recognition of the image to be recognized.

2. The identification method according to claim 1, wherein: The performing recognition of the to-be-recognized image based at least on a portion of the network modules among the plurality of network modules close to the input side of the neural network model comprises: For any network module among the multiple network modules, in response to the network module receiving the image to be identified or one of the feature maps output by a previous network module, the network module outputs a predicted classification of the image to be identified and its confidence; In response to the confidence being greater than a preset threshold, determining that the prediction is classified as a recognition result.

3. The identification method according to claim 2, further comprising: For any network module among the multiple cascaded network modules that is not located at the output end of the neural network model, in response to the confidence level being less than the preset threshold, the feature map output by the network module is input to the next network module.

4. The identification method according to claim 3, further comprising: For the network module located at the output end of the neural network model, in response to the confidence being less than the preset threshold, the predicted classification is determined to be a recognition result.

5. A method for training a neural network model for image recognition, wherein: The neural network model includes a plurality of cascaded network modules, the plurality of cascaded network modules include at least one super network module, and the training method includes: Acquire a plurality of first training sets, wherein each of the plurality of first training sets includes a plurality of first training samples and their true classifications; According to a first preset mode, determining a first training set corresponding to each network module in the plurality of cascaded network modules; and Using the first training set corresponding to each network module in the plurality of network modules, the parameters of the network module are adjusted, wherein: For each hypernetwork module in the at least one hypernetwork module and its corresponding first training set, In response to each of the first training samples in the first training set being input into the neural network model, obtaining a predicted classification and a first calculation duration of the first training sample output by the hypernetwork module, and Adjusting the parameters of the hypernetwork module based on the true classification, the predicted classification, and the first calculation duration of each first training sample in the plurality of first training samples in the first training set; Each of the at least one super network module includes at least two sub-networks, one of the at least two sub-networks of the super network module is determined as a selected sub-network, and the selected sub-network is used to perform recognition of the image to be recognized.

6. The training method according to claim 5, wherein: The plurality of first training sets have at least two different recognition difficulty levels, The determining, according to the first preset mode, a first training set corresponding to each of the plurality of cascaded network modules comprises: According to the position of each network module in the neural network model, determining a first training set corresponding to each network module among the plurality of network modules, Among them, for any two network modules among the multiple network modules, the recognition difficulty level of the training set corresponding to the network module close to the input side of the neural network model is not greater than the recognition difficulty level of the training set corresponding to the other network module.

7. The training method according to claim 5 or 6, wherein: The neural network model also includes a predictor corresponding to each of the at least one super network module, The training method further comprises: For each of the at least one hypernetwork module, after adjusting the parameters of the hypernetwork module, Get the second training set; The parameters of the predictor corresponding to the super network module are adjusted using the second training set so that the predictor can determine that one of the at least two sub-networks of the super network module is the selected sub-network based on the feature map output by the previous network module of the super network module.

8. The training method according to claim 5 or 6, wherein: The training method further comprises: For each of the at least one hypernetwork module, after adjusting the parameters of the hypernetwork module, Get the test set; Using the test set, one of the at least two sub-networks of the super network module is determined as a selected sub-network.

9. The training method according to claim 5, wherein: The adjusting the parameters of each network module by using the first training set corresponding to the network module includes: For any two network modules among the multiple network modules, the parameters of the network modules close to the input side of the neural network model are first adjusted.

10. An image recognition device, comprising: A neural network model, comprising a plurality of cascaded network modules, wherein the plurality of cascaded network modules include at least one super network module, each of the at least one super network module includes at least two sub-networks, and the neural network model further includes a predictor corresponding to each of the at least one super network module; An input unit, configured to input the image to be recognized into the neural network model; The neural network model is configured to perform recognition of the image to be recognized based on at least some of the network modules in the plurality of network modules that are close to the input side of the neural network model, wherein: For any hypernetwork module in the at least one hypernetwork module and its corresponding predictor, When the super network module receives the feature map output by the previous network module, the feature map is input into the predictor corresponding to the super network module to obtain the predicted value output by the predictor. Based on the predicted value, determining one of the at least two sub-networks of the super-network module as a selected sub-network, and In the super network module, the selected sub-network is used to perform recognition of the image to be recognized.

11. The identification device according to claim 10, wherein: The neural network model is further configured to output, for any network module among the multiple network modules, a predicted classification of the image to be identified and its confidence level in response to the network module receiving the image to be identified or one of the feature maps output by a previous network module; The identification device also includes: The determination unit is configured to determine that the predicted classification is a recognition result in response to the confidence level being greater than a preset threshold.

12. The identification device according to claim 11, wherein the determining unit is further configured to: For any network module among the multiple cascaded network modules that is not located at the output end of the neural network model, in response to the confidence level being less than the preset threshold, the feature map output by the network module is input to the next network module.

13. A training device for a neural network model for image recognition, wherein: The neural network model includes a plurality of cascaded network modules, the plurality of cascaded network modules include at least one super network module, and the training device includes: A first acquisition unit is configured to acquire a plurality of first training sets, wherein each of the plurality of first training sets includes a plurality of first training samples and their true classifications; A first determining unit is configured to determine a first training set corresponding to each network module in the plurality of cascaded network modules according to a first preset mode; and The first adjustment unit is configured to adjust the parameters of each network module by using the first training set corresponding to each network module in the plurality of network modules, wherein: For each hypernetwork module in the at least one hypernetwork module and the first training set corresponding thereto, In response to each of the first training samples in the first training set being input into the neural network model, obtaining a predicted classification and a first calculation duration of the first training sample output by the hypernetwork module, and Adjusting the parameters of the hypernetwork module based on the true classification, the predicted classification, and the first calculation duration of each first training sample in the plurality of first training samples in the first training set; Each of the at least one super network module includes at least two sub-networks, one of the at least two sub-networks of the super network module is determined as a selected sub-network, and the selected sub-network is used to perform recognition of the image to be recognized.

14. The training device according to claim 13, wherein: The plurality of first training sets have at least two different recognition difficulty levels, The determining unit is further configured to determine a first training set corresponding to each network module among the plurality of network modules according to a position of the network module in the neural network model. Among them, for any two network modules among the multiple network modules, the recognition difficulty level of the training set corresponding to the network module close to the input side of the neural network model is not greater than the recognition difficulty level of the training set corresponding to the other network module.

15. A training device according to claim 13 or 14, wherein: The neural network model also includes a predictor corresponding to each of the at least one super network module, The training device further includes a second acquisition unit and a second adjustment unit, wherein: For each of the at least one hypernetwork module, after adjusting the parameters of the hypernetwork module, The second acquisition unit is configured to acquire a second training set; The second adjustment unit is configured to adjust the parameters of the predictor corresponding to the super network module using the second training set so that the predictor can determine that one of the at least two sub-networks of the super network module is the selected sub-network based on the feature map output by the previous network module of the super network module.

16. A training device according to claim 13 or 14, wherein: The training device further includes a third acquisition unit and a second determination unit, wherein: For each of the at least one hypernetwork module, after adjusting the parameters of the hypernetwork module, The third acquisition unit is configured to acquire a test set; The second determining unit is configured to use the test set to determine that one of the at least two sub-networks of the super-network module is a selected sub-network.

17. The training device according to claim 13, wherein: The first adjustment unit is further configured to: For any two network modules among the multiple network modules, the parameters of the network modules close to the input side of the neural network model are first adjusted.

18. A computer device comprising: a memory, a processor and a computer program stored on the memory, The processor is configured to execute the computer program to implement the steps of the method according to any one of claims 1 to 9.

19. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

20. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.