Image recognition method and model training method, device and equipment, and storage medium

Abstract

Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to an image recognition method and model training method, device and equipment, and storage medium, using an image recognition model, the image recognition model comprising a plurality of sub-recognition models cascaded, and a decision maker added between two adjacent sub-recognition models. In this case, in the image recognition method, the decision maker is used to process image feature information extracted by a current target sub-recognition model and an image recognition result output by the current target sub-recognition model, to score the image recognition result output by the current sub-recognition model, and in the case that the score value meets a standard, the current image recognition result is determined as a final image recognition result, and the process of sending a to-be-recognized image into a next sub-recognition model is stopped. The addition of the decision maker can improve the image recognition accuracy and efficiency.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of image recognition, and more specifically, the embodiments of this disclosure relate to image recognition methods and model training methods, apparatus, devices and storage media. Background Technology

[0002] This section is intended to provide background or context for embodiments of this disclosure, and the description herein is not acknowledged as related technology simply because it is included in this section.

[0003] Image recognition refers to the technology of using computers to process, analyze, and understand images in order to identify targets and objects of various patterns. It is a practical application of deep learning algorithms.

[0004] Accurately identifying a target from a massive, noisy, and random collection of internet images is a huge challenge. In one related technology, to increase the accuracy of image recognition, a multi-model cascade approach is often used. That is, the image recognition model consists of multiple sub-recognition models. After the same image is fed into the first sub-recognition model for recognition, the prediction scores of each sub-recognition model are compared with a pre-trained judgment threshold to determine whether to feed it into the next sub-recognition model for recognition.

[0005] Specifically, if the prediction score of the current sub-recognition model is not less than the corresponding judgment threshold, the image recognition result of the current sub-recognition model is used as the final recognition result of the image. If the score of the current sub-recognition model is less than the corresponding judgment threshold, the image is sent to the next sub-recognition model for recognition.

[0006] In this way, the initial model can be used to filter out simple and easily identifiable images, and then images that are difficult to analyze and identify can be left for recognition by a larger model in the later stages. Summary of the Invention

[0007] In this context, embodiments of the present invention aim to provide an image recognition method, a model training method, apparatus, device, and storage medium.

[0008] According to one aspect of this disclosure, an image recognition method based on an image recognition model is provided. The image recognition model includes multiple cascaded sub-recognition models and a decision unit connected between two adjacent sub-recognition models. The image recognition method includes:

[0009] In the case where the target sub-recognition model is determined sequentially among multiple sub-recognition models, and there is a next sub-recognition model to be determined after the current target sub-recognition model, the following steps are performed:

[0010] Recognition steps: Input the image to be recognized into the current target sub-recognition model so that the current target sub-recognition model outputs the first image feature information, and perform image recognition based on the first image feature information to obtain the first image recognition result;

[0011] Decision-making steps: The decision-maker between the current target sub-recognition model and the next sub-recognition model to be determined is used as the target decision-maker. The first image feature information and the first image recognition result are input into the target decision-maker so that the target decision-maker scores the first image recognition result to obtain a score value.

[0012] If the score meets the standard, the first image recognition result is determined as the final image recognition result, and the determination of the next sub-recognition model after the current target sub-recognition model as the target sub-recognition model is stopped.

[0013] According to another aspect of this disclosure, a method for training an image recognition model is also provided. The image recognition model to be trained includes multiple cascaded sub-recognition models and a decision unit to be trained connected between two adjacent sub-recognition models. The training method for the image recognition model employs the following steps to train each decision unit to be trained:

[0014] For the current decision maker to be trained, the sample image is input into the two adjacent sub-recognition models before and after the current decision maker to be trained, so that the sub-recognition model before the current decision maker to be trained outputs the second image feature information and predicts the second image recognition result based on the second image feature information, and the sub-recognition model after the current decision maker to be trained outputs the third image recognition result.

[0015] The second image feature information and the second image recognition result are input into the current decision-maker to be trained, so that the second image recognition result is scored by the current decision-maker to be trained to obtain a score value;

[0016] The correctness of the second image recognition result is checked based on the real results of the sample images to obtain the first check result, and the correctness of the third image recognition result is checked based on the real results to obtain the second check result. The reward value for the score value is obtained according to the consistency relationship between the first check result and the second check result. The loss function is calculated according to the score value and the reward value, and the current decision-maker to be trained is trained using the loss function.

[0017] According to another aspect of this disclosure, an image recognition device based on an image recognition model is also provided. The image recognition model includes multiple cascaded sub-recognition models and a decision unit connected between two adjacent sub-recognition models.

[0018] In a scenario where a target sub-recognition model is sequentially determined among multiple sub-recognition models, and there is a next sub-recognition model to be determined after the current target sub-recognition model, the image recognition device includes the following modules:

[0019] The first image recognition module inputs the image to be recognized into the current target sub-recognition model so that the current target sub-recognition model outputs the first image feature information, and performs image recognition based on the first image feature information to obtain the first image recognition result;

[0020] The first decision module uses the decision-maker between the current target sub-recognition model and the next sub-recognition model to be determined as the target decision-maker, and inputs the first image feature information and the first image recognition result into the target decision-maker to score the first image recognition result and obtain a score value.

[0021] The determination module, if the score value meets the standard, determines the first image recognition result as the final image recognition result and stops determining the next sub-recognition model after the current target sub-recognition model as the target sub-recognition model.

[0022] According to another aspect of this disclosure, there is also a training apparatus for an image recognition model, wherein the image recognition model to be trained includes multiple cascaded sub-recognition models and a decision unit to be trained connected between two adjacent sub-recognition models; the training apparatus for the image recognition model includes the following modules for training each decision unit to be trained:

[0023] The second image recognition module, for the current decision maker to be trained, inputs the sample image into the two sub-recognition models adjacent to the current decision maker before and after the current decision maker, so that the sub-recognition model adjacent to the current decision maker before the current decision maker outputs the second image feature information, and predicts the second image recognition result based on the second image feature information, and the sub-recognition model adjacent to the current decision maker after the current decision maker outputs the third image recognition result.

[0024] The second decision module inputs the second image feature information and the second image recognition result into the current decision-maker to be trained, so as to score the second image recognition result through the current decision-maker to be trained to obtain a score value.

[0025] The training module performs a correctness check on the second image recognition result based on the real results of the sample images to obtain a first check result, and performs a correctness check on the third image recognition result based on the real results to obtain a second check result. It also obtains a reward value for the score value based on the consistency relationship between the first check result and the second check result, calculates a loss function based on the score value and the reward value, and uses the loss function to train the current decision-maker to be trained.

[0026] According to one aspect of this disclosure, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements: the image recognition method based on the image recognition model described above, or the training method for the image recognition model.

[0027] According to one aspect of this disclosure, an electronic device is provided, comprising:

[0028] Processor; and

[0029] Memory is used to store the processor's executable instructions;

[0030] The processor is configured to execute the aforementioned image recognition method based on the image recognition model, or the training method of the image recognition model, by executing executable instructions.

[0031] According to the image recognition method, model training method, apparatus, device, and storage medium of this disclosure, an image recognition model is used, which includes multiple cascaded sub-recognition models, and a decision unit is added between two adjacent sub-recognition models. In this case, in the image recognition method, the decision unit processes the image feature information extracted by the current target sub-recognition model and the image recognition result output by the current sub-recognition model to score the image recognition result output by the current sub-recognition model. If the score value meets the standard, the current image recognition result is determined as the final image recognition result, and the process of sending the image to be recognized to the next sub-recognition model is stopped.

[0032] In this embodiment, the addition of the decision-maker not only eliminates the cost of manually setting thresholds, but also the decision-making process changes dynamically with the number of predictions, avoiding false detections and missed detections caused by the sensitivity of prediction accuracy to thresholds, thereby improving the accuracy and efficiency of image recognition. Attached Figure Description

[0033] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0034] Figure 1 One of the flowcharts of an image recognition method according to an embodiment of the present disclosure is illustrated schematically;

[0035] Figure 2 A schematic diagram illustrating the principle of the image recognition method based on the image recognition model according to embodiments of the present disclosure is shown.

[0036] Figure 3 A schematic diagram of the sub-recognition model in the image recognition model according to an embodiment of the present disclosure is shown.

[0037] Figure 4 A schematic diagram of the structure of a decision-maker in an image recognition model according to an embodiment of the present disclosure is shown.

[0038] Figure 5 A flowchart illustrating a training method for an image recognition model according to an embodiment of the present disclosure is shown schematically;

[0039] Figure 6 A block diagram of an image recognition apparatus based on an image recognition model according to an embodiment of the present disclosure is shown schematically;

[0040] Figure 7 A block diagram schematically illustrates a training apparatus for an image recognition model according to an embodiment of the present disclosure;

[0041] Figure 8 A schematic diagram of a storage medium according to an embodiment of the present disclosure is shown;

[0042] Figure 9 A block diagram of an electronic device according to a disclosed embodiment is shown schematically.

[0043] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0044] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0045] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0046] According to embodiments of this disclosure, an image recognition and model training method, apparatus, device, and storage medium are provided.

[0047] In this document, any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0048] The principles and spirit of this disclosure are explained in detail below with reference to several representative embodiments. Invention Overview

[0050] Analysis of related technologies reveals that in image recognition schemes employing multi-model cascades, the only way to determine whether to feed the image into a subsequent sub-model is to check if the current sub-model's prediction score for the image exceeds a threshold. This necessitates testing on a large amount of non-training data, followed by manual integration and selection to confirm the threshold for each model. This method is not only time-consuming and labor-intensive, but also suffers from the sensitivity of random data to thresholds; low thresholds result in numerous false detections, while high thresholds lead to some images not being detected by the model. Therefore, these technologies suffer from low image recognition accuracy.

[0051] Based on the aforementioned technical problems, this disclosure proposes an improved image recognition method based on an image recognition model. The inventive idea is that the image recognition model is obtained by cascading multiple sub-recognition models, with a decision unit added between adjacent sub-recognition models. In this case, the decision unit processes the image feature information extracted by the current sub-recognition model and the output image recognition result to score the image recognition result output by the current sub-recognition model. If the score meets the standard, the current image recognition result is determined as the final image recognition result, and the process of feeding the image into the next sub-recognition model is stopped.

[0052] In this embodiment, the addition of the decision-maker not only eliminates the cost of manually setting thresholds, but also the decision-making process changes dynamically with the number of predictions, avoiding false detections and missed detections caused by the sensitivity of prediction accuracy to thresholds, thereby improving the accuracy and efficiency of image recognition.

[0053] After introducing the basic principles of the present invention, various non-limiting embodiments of the present invention will be described in detail below.

[0054] Exemplary methods

[0055] The image recognition method and image recognition model training method according to exemplary embodiments of the present disclosure will now be described with reference to the accompanying drawings. The image recognition model used in the image recognition method may be obtained using the training method shown in the exemplary embodiments of the present disclosure, or it may be obtained using other training methods.

[0056] For ease of understanding, the following explanations are provided for several terms used in the embodiments of this disclosure.

[0057] Static recognition

[0058] Static recognition refers to a neural network having a static architecture. After an image is input into the neural network, the calculation method and process are statically fixed each time. Only one or more specified positions can be set to output results, and the position and number of output results are fixed each time and do not change with the input.

[0059] Dynamic recognition

[0060] Dynamic recognition refers to a neural network that is a dynamic architecture with dynamic mechanisms within or between networks. As the input changes, the calculation method or process changes, and the position or number of outputs may also change, so the output result is not fixed.

[0061] The following is combined Figure 1 and Figure 2 This document describes an image recognition method based on an image recognition model according to exemplary embodiments of the present disclosure.

[0062] like Figure 1 As shown, the image recognition model includes:

[0063] Multiple cascaded sub-recognition models, such as sub-recognition models 1, 2...N;

[0064] Decision generators are connected between two adjacent sub-recognition models, such as decision generator 1, 2...N-1, where decision generator 1 is located between sub-recognition models 1 and 2, and the other decision generators can be set up similarly.

[0065] The image recognition method of exemplary embodiments of this disclosure includes:

[0066] The target sub-recognition model is determined sequentially among multiple sub-recognition models. If the current target sub-recognition model has been determined and there is a next undetermined sub-recognition model following it, then execution is performed. Figure 1 The steps shown are as follows:

[0067] Recognition step 110: Input the image to be recognized into the current target sub-recognition model so that the current target sub-recognition model outputs the first image feature information, and performs image recognition based on the first image feature information to obtain the first image recognition result;

[0068] Decision step 120: The decision-maker between the current target sub-recognition model and the next sub-recognition model to be determined is used as the target decision-maker. The first image feature information and the first image recognition result are input into the target decision-maker so that the target decision-maker scores the first image recognition result to obtain a score value.

[0069] If the score meets the standard, the first image recognition result is determined as the final image recognition result, and the determination of the next sub-recognition model after the current target sub-recognition model as the target sub-recognition model is stopped.

[0070] In this embodiment, the decision-maker is used to score the target recognition result of the current target sub-recognition model based on the image feature information and image recognition result extracted by the current target sub-recognition model. Since the target sub-recognition model yields different image recognition results for different images to be recognized, the decision-maker's score for the target sub-recognition model is dynamic and not fixed each time, thus enabling a dynamic image recognition scheme.

[0071] Therefore, in the exemplary embodiments of this disclosure, the addition of the decision-maker not only saves the cost of manually setting the threshold, but also its decision-making process changes dynamically with the number of predictions, avoiding false detection and missed detection problems caused by the sensitivity of prediction accuracy to the threshold, thereby improving the accuracy and efficiency of image recognition.

[0072] like Figure 2 As shown, this disclosure provides a specific implementation method based on... Figure 1 The image recognition method of the image recognition model shown specifically includes:

[0073] Starting with the first sub-recognition model 1, it is identified as the current target sub-recognition model, and execution is performed using this first sub-recognition model 1. Figure 1 The identification step 110 is shown;

[0074] Next, the first image feature information and the first image recognition result output from recognition step 110 are input into the first decision-maker 1. The first decision-maker 1 serves as the target decision-maker, and execution is carried out through the first decision-maker 1. Figure 1 Decision step 120 is shown;

[0075] Next, the judgment step 210 is executed to determine whether the score value of the first image recognition result output by the decision step 120 meets the standard, so as to further determine whether to stop image recognition based on the judgment result;

[0076] If so, then stop identifying the second sub-recognition model 2 after the first sub-recognition model 1 as the target sub-recognition model, and determine the first image recognition result as the final image recognition result;

[0077] If not, meaning the score does not meet the standard, then the second sub-recognition model 2 is determined as the next target sub-recognition model, and the process is repeated based on this next target sub-recognition model. Figure 1 The identification step 110 and decision step 120 are shown.

[0078] Specifically, determining the first image recognition result as the final image recognition result includes:

[0079] Step 220: Determine whether the image recognition score output by the first sub-recognition model 1 is greater than the second threshold. If so, determine that the image to be recognized is a target class image; otherwise, determine that the image to be recognized is a non-target class image.

[0080] Specifically, the first image recognition result is compared with a second threshold;

[0081] If the first image recognition result is greater than the second threshold, then the final image recognition result is determined to be an image of the target class.

[0082] If the first image recognition result is not greater than the second threshold, then the final image recognition result is determined to be an image of the target class not recognized.

[0083] Therefore, by comparing the first image recognition result with the second threshold, the recognition result of whether the image to be recognized is a target class image is output. The second threshold is the critical value for target class image recognition of the image to be recognized.

[0084] For the execution logic of other sub-recognition models, please refer to [the relevant documentation / reference]. Figure 1 and Figure 2 The content shown will not be repeated here.

[0085] In an exemplary embodiment of this disclosure, further, for the last sub-recognition model N, when it is determined to be the target in the recognition model, since no decision unit is connected after it, the first image recognition result output by the last sub-recognition model N can be determined as the final image recognition result.

[0086] As can be seen from the above exemplary implementation, the feature vector and prediction score output by the previous sub-recognition model k are used as input, and whether to stop the image recognition of the next sub-recognition model k+1 is used as output (for example, 0 indicates to continue the recognition of the next sub-recognition model, and 1 indicates to stop the recognition process and output the prediction score). Therefore, the earlier sub-recognition models can first filter out simple and easily identifiable images, and then leave the images that are difficult to analyze and identify to subsequent sub-recognition models for recognition, thus balancing image recognition efficiency and accuracy.

[0087] In the exemplary embodiments of this disclosure, the choice of sub-recognition models can be arbitrary, such as MobileNet_v2, ResNet50, ViT-B, etc. However, in order to use small and fast initial models to first filter out simple and easy-to-recognize images, and then leave the images that are difficult to analyze and recognize to subsequent larger models for recognition, the selection of cascaded models follows certain rules:

[0088] That is, the former model has lower accuracy than the latter model, but the prediction time is faster. In other words, as the prediction process continues, the model's prediction becomes more and more time-consuming, but the accuracy becomes more and more accurate.

[0089] In this context, within image recognition models, the network structure of multiple cascaded sub-recognition models becomes increasingly refined, following a cascading relationship from front to back. That is, the later the sub-recognition model, the higher its network structure refinement. The metrics for evaluating network structure refinement include the number of network layers, the number of channels in each layer, and / or the resolution of the feature maps within the network. The number of network layers represents network depth, and the number of channels in each layer represents network width. Therefore, the higher the network depth, width, and / or resolution, the higher the refinement of the network structure.

[0090] In one alternative implementation, Figure 3 Taking ResNet50 as an example, each sub-recognition model may include a backbone network, followed by fully connected layers and a normalized exponential function softmax. The backbone network can be a neural network.

[0091] In the prediction stage, the preprocessed image to be recognized is fed into a single sub-recognition model k (k = 1, 2, ..., N) in the cascaded model-based image recognition model. The image feature vector (Feature) is extracted through its backbone network, and then the prediction score (Score) is calculated through a fully connected layer and subsequent softmax. The Feature and Score represent the image recognition scores of the first image feature information extracted by the sub-recognition model and the first image recognition result predicted by the sub-recognition model, respectively.

[0092] In the exemplary embodiments of this disclosure, after obtaining the original image to be recognized, image preprocessing can be performed first, which mainly includes data normalization and image scaling. Specifically, to ensure a balance between speed and accuracy, the image to be recognized is usually scaled before being fed into the sub-recognition model. The specific scaling scale depends on the actual task requirements, typically defaulting to 224 pixels × 224 pixels, but other scaling resolutions can be selected as needed. Normalization is performed to ensure consistency with the data distribution during model training.

[0093] In this way, the image to be identified after image preprocessing can be used as input for any subsequent target sub-identification model.

[0094] In an exemplary embodiment of this disclosure, the above-mentioned determination of whether the score value meets the standard can specifically be to determine the size of the score value and the first threshold. If the score value is not lower than the first threshold, the score value is determined to meet the standard; otherwise, if the score value is lower than the first threshold, the score value is determined to fail to meet the standard.

[0095] Compared to the judgment thresholds in related technologies, which require constant training and adjustment, the first threshold is an inherent parameter of the decision-maker itself, which is independent of the decision-maker's input data. Therefore, it does not require a large amount of non-training data for testing, nor does it require frequent adjustments to the threshold through manual integration and selection, thus greatly reducing the consumption of data and manpower.

[0096] In an exemplary embodiment of this disclosure, the first image recognition result is represented as a probability value, for example, characterizing the probability that the image to be recognized is a target class image. In this case, the step 220 described above, which scores the first image recognition result using a target decision unit to obtain a score value, may include:

[0097] The target decision unit weights the first image feature information by multiplication based on probability values ​​to obtain the target image features, and predicts the score value of the first image recognition result based on the target image features.

[0098] In this embodiment, the target image feature integrates the first image feature information and the image recognition score represented by the first image recognition result, which can characterize the degree of contribution of each image feature in the first image feature information to the above probability value. In this way, the target decision-maker predicts the score value of the first image recognition result through the contribution relationship between the first image feature information and the probability value.

[0099] In an exemplary embodiment of this disclosure, the target image features are obtained by weighting the first image feature information based on probability values ​​using a target decision unit in a multiplicative manner, including:

[0100] The target decision unit integrates the features of the first image to obtain integrated image features.

[0101] The target image features are obtained by multiplying the integrated image features based on probability values ​​using a target decision unit.

[0102] As mentioned above, feature integration is a further extraction of the feature information of the first image, which can remove image features that do not contribute much to the recognition result of the first image. This makes the subsequent prediction and scoring process more efficient, thereby improving the scoring decision efficiency of the target decision-maker.

[0103] In similar exemplary embodiments of this disclosure, the integrated image features are weighted by multiplication based on probability values. Specifically, each vector element in the feature vector of the integrated image features is multiplied by the probability value to obtain the target image features, which exist in the form of feature vectors.

[0104] In exemplary embodiments of this disclosure, such as Figure 4The model structure of the decision-maker shown includes two sets of sequentially connected feature processing units, each set of feature processing units including a first fully connected layer (FC), a batch normalization layer (BN), and a nonlinear layer (ReLU) cascaded in sequence.

[0105] The nonlinear layer uses the rectified linear unit (ReLU), also known as the modified linear unit, which is a commonly used activation function in artificial neural networks. It typically refers to nonlinear functions represented by the ramp function and its variants. However, this disclosure does not exclude the possibility of using other nonlinear functions in the nonlinear layer.

[0106] In this case, the input to the first set of feature processing units is the first image feature information, which is a feature vector. The first set of feature processing units performs feature integration until integrated image features are obtained.

[0107] Specifically, the feature vector of the first image feature information has a size of B*C, where B is the batch size and C is the length of the feature vector extracted by the current target sub-recognition model. The first set of feature processing units performs weighted integration of the feature vector of the first image feature information at the feature level, which can reduce the feature vector length to C / 8 or other ratios.

[0108] The input to the second set of feature processing units is the output of the first set of feature processing units and the prediction score of the first image recognition result. The score is used to multiply the output of the first set of feature processing units to obtain the target image features.

[0109] In this way, after passing through these two sets of feature processing units, the feature vector length of the first image feature information is reduced from C to C / 8, and then to C / 32, thereby reducing feature information redundancy and improving inference speed.

[0110] Optionally, the decision-maker may not be limited to two sets of feature processing units, wherein the second part includes only the last set of feature processing units, while the first part may include one or more other sets of feature processing units other than the last set of feature processing units, such that the decision-maker may include at least two sets of sequentially connected feature processing units, which are divided into a first part and a second part based on the connection relationship, and each set of feature processing units includes a first fully connected layer, a batch normalization layer and a nonlinearization layer connected in sequence.

[0111] In this case, the target decision-maker integrates the first image feature information to obtain integrated image features, including:

[0112] The first image feature information is input into the first part, and the first image feature information is integrated sequentially by each feature processing unit in the first part until the integrated image features are obtained.

[0113] The target image features are obtained by multiplying the integrated image features based on probability values ​​using a target decision unit, including:

[0114] The probability value and the integrated image features are input into the second part, so that the final set of feature processing units in the second part can multiply the integrated image features based on the probability value to obtain the target image features.

[0115] In exemplary embodiments of this disclosure, such as Figure 4 As shown, the decision-maker also includes a second fully connected layer (FC) connected after the second part and an activation function sigmoid. In this case, predicting the score of the first image recognition result based on the target image features can include:

[0116] The target image features are scored and predicted using a second fully connected layer.

[0117] The predicted values ​​are nonlinearized by an activation function to obtain the score of the first image recognition result.

[0118] That is, the second fully connected layer is used as the prediction layer, and then the prediction score of the second fully connected layer is non-linearized by Sigmoid so that the prediction score falls in [0, 1], and the score value prob of the first image recognition result is output.

[0119] In one alternative embodiment, since prob is nonlinearized through a Sigmoid layer and its score range is [0,1], 0.5 can be used as the first threshold for division: when prob>0.5, 1 is output, that is, the recognition process stops and the final image recognition result is output;

[0120] When prob <= 0.5, output 0, that is, continue to the next target sub-identification model identification step.

[0121] The first threshold of 0.5 is one example. In other embodiments, the first threshold can be selected from other values, which can be adjusted as needed and are not limited here.

[0122] This disclosure also provides an exemplary embodiment of a method for training the above-described image recognition model, wherein the image recognition model to be trained may include multiple cascaded sub-recognition models and a decision generator to be trained connected between two adjacent sub-recognition models.

[0123] In this case, such as Figure 5 As shown, each decision generator to be trained is trained using the following steps:

[0124] Step 510: For the current decision maker to be trained, input the sample image into the two adjacent sub-recognition models before and after the current decision maker to be trained, so that the sub-recognition model before the current decision maker to be trained outputs the second image feature information and predicts the second image recognition result based on the second image feature information, and the sub-recognition model after the current decision maker to be trained outputs the third image recognition result.

[0125] Step 520: Input the second image feature information and the second image recognition result into the current decision-maker to be trained, so as to score the second image recognition result by the current decision-maker to be trained to obtain a score value;

[0126] Step 530: Perform a correctness check on the second image recognition result based on the real result of the sample image to obtain a first check result, and perform a correctness check on the third image recognition result based on the real result to obtain a second check result. Obtain a reward value for the score value based on the consistency relationship between the first check result and the second check result, calculate the loss function based on the score value and the reward value, and use the loss function to train the current decision-maker to be trained.

[0127] The model training method exemplified above is based on the principle of reinforcement learning to train each decision-maker. Reinforcement learning involves an agent learning through trial and error, using rewards obtained from interacting with the environment to guide behavior. The goal is to maximize the agent's reward. Reinforcement learning differs from supervised learning in connectionist learning primarily in its reinforcement signals. In reinforcement learning, the reinforcement signals provided by the environment evaluate the quality of the actions performed.

[0128] Specifically, in the exemplary embodiments of this disclosure, a single decision-maker is trained using reinforcement learning. The decision-maker is referred to as the agent, and the decision-making process is viewed as a Markov decision process (MDP), where M = (S, A, R), where:

[0129] State Set S: The feature vector and prediction score of the target image extracted by the recognition model k, i.e., s i =(Feature) i Scorei ), where i represents the i-th sample;

[0130] Action Set A: A has 2 actions, A = {0, 1}, where 0 indicates continuing the recognition of the next recognition model k+1, and 1 indicates stopping the recognition process;

[0131] Reward R: Constructing a reward mechanism is crucial for successfully training a reinforcement learning agent. A poor reward mechanism can lead to training non-convergence and the agent's inability to make effective decisions. To ensure stable training of the decision-maker, an effective right-or-wrong reward mechanism should be constructed, which determines the reward value solely based on whether the predictions of the two preceding and succeeding sub-models (sub-model k and sub-model k+1) are correct.

[0132] Since a single decision-maker is only responsible for deciding whether to continue recognizing the next sub-recognition model, the outputs of models other than sub-recognition model k and sub-recognition model k+1 are all interferences to the training of the decision-maker. In the experiment, it was found that adding more sub-recognition model outputs would lead to unstable training, while focusing only on the outputs of the two sub-recognition models before and after the current decision-maker training makes the decision-maker training converge stably.

[0133] Therefore, the principle for constructing a reward mechanism is:

[0134] ① Allow the recognition process to stop earlier and more correctly at the sub-recognition model k;

[0135] ② Only when sub-recognition model k makes a mistake should sub-recognition model k+1 continue to make a mistake.

[0136] In this case, the reward design is as follows:

[0137]

[0138]

[0139] Where r is the reward value and m is the coefficient. In the experiment, it was found that r generally takes the value (0.5, 1] ​​and m takes the value [5, 10] for better results. However, this is not the only limitation on the range of values ​​of r and m, and can be adjusted according to different application scenarios.

[0140] In this model, the image recognition result of the correctly represented corresponding sub-model is consistent with the true result, i.e., the prediction is correct. In the model, the image recognition result of the incorrectly represented corresponding sub-model is inconsistent with the true result, i.e., the prediction is incorrect.

[0141] In this case, the reward value for awarding the score based on the consistency relationship between the first and second test results can include:

[0142] By using the first and second test results to look up a table, the consistency relationship between the first and second test results and the reward value matched by the consistency relationship can be obtained.

[0143] The training of each sub-recognition model and the decision-maker can be sequential, that is, each sub-recognition model is trained first, and then each decision-maker is trained independently using policy gradient. Alternatively, each sub-recognition model and its adjacent and subsequent decision-maker can be jointly trained as a training unit.

[0144] Using a cascaded training method, pre-training the sub-recognition models allows for individual tuning of hyperparameters, enabling each sub-recognition model to achieve its best results. However, joint training not only requires a significant increase in the number of hyperparameters to be tuned, but also makes it difficult for each model to achieve its optimal training effect.

[0145] Therefore, before inputting the sample image into the current decision-maker to be trained and before the two adjacent sub-recognition models, the two adjacent sub-recognition models before and after the current decision-maker to be trained are trained to obtain the two trained sub-recognition models.

[0146] The training of a single decision-maker is as follows:

[0147] After obtaining the prediction outputs of the two sub-recognition models, the action is obtained by randomly sampling between [0,1] according to the score prob as the probability value. Then, after calculating the reward, the loss is:

[0148] loss = -log(prob) * reward

[0149] The loss function takes the score value (prob) and reward value (reward) of the decision-maker to be trained as inputs to calculate the loss of the decision-maker, so as to achieve the purpose of reinforcement training.

[0150] In an exemplary embodiment of this disclosure, the second image recognition result is represented as a probability value;

[0151] In this case, the second image recognition result is scored using the current decision-maker to obtain a score value, including:

[0152] The current decision maker to be trained calculates the target image features by multiplying the second image feature information based on probability values, and then predicts the score of the second image recognition result based on the target image features.

[0153] Optionally, the current decision generator to be trained further includes multiplying the second image feature information based on probability values ​​to obtain the target image features, and also includes:

[0154] The second image feature information is integrated by the decision-maker to be trained to obtain integrated image features;

[0155] The target image features are obtained by multiplying the integrated image features based on probability values ​​using the decision-maker to be trained.

[0156] In the exemplary embodiments of this disclosure, combined with Figure 4 As shown, each decision maker includes at least two sets of sequentially connected feature processing units. The at least two sets of sequentially connected feature processing units are divided into a first part and a second part based on the connection relationship. The second part includes only the last set of feature processing units. The first part includes all feature processing units except the last set of feature processing units. Each set of feature processing units includes a first fully connected layer, a batch normalization layer and a nonlinearization layer connected in sequence.

[0157] The second image feature information is integrated using the current decision maker to be trained, to obtain integrated image features, including:

[0158] The second image feature information is input into the first part, and the feature processing units in the first part sequentially integrate the second image feature information until the integrated image features are obtained.

[0159] The target image features are obtained by multiplying the integrated image features based on probability values ​​using the current decision-maker to be trained, including:

[0160] The probability value and the integrated image features are input into the second part, so that the final set of feature processing units in the second part can multiply the integrated image features based on the probability value to obtain the target image features.

[0161] Optionally, the decision maker further includes a second fully connected layer and an activation function connected after the second part; predicting a score value for the second image recognition result based on the target image features, including:

[0162] The target image features are scored and predicted using a second fully connected layer.

[0163] The predicted values ​​are nonlinearized by an activation function to obtain the score of the second image recognition result.

[0164] In the exemplary embodiments of this disclosure, based on the cascading order of the sub-recognition models from front to back, the sub-recognition models in the image recognition model are cascaded sequentially from coarse to fine according to the network structure.

[0165] As can be seen from the above, the exemplary image recognition method and model training method disclosed herein can be used to identify prohibited images, wherein prohibited images are images containing prohibited targets or objects. Therefore, the identification of prohibited images refers to the identification of prohibited targets or objects in the images.

[0166] In this scenario, using the exemplary embodiments described above, the autonomous decision-based dynamic recognition scheme for prohibited images constructs a recognition system of multiple cascaded models and inserts a decision-maker into the cascaded models. This allows the entire recognition system to dynamically choose whether to continue recognizing during the recognition process, avoiding the sensitivity and other problems of traditional threshold methods. The simple and effective model architecture and the design of the right and wrong reward mechanism make the trained decision-maker stable and effective, thereby improving the accuracy and recall capability of the recognition service system.

[0167] Exemplary device

[0168] After introducing the image recognition method and image recognition model training method according to exemplary embodiments of this disclosure, the following will refer to... Figure 6 An image recognition apparatus based on an image recognition model according to an exemplary embodiment of the present disclosure will be described.

[0169] The image recognition model includes multiple cascaded sub-recognition models and a decision unit connecting two adjacent sub-recognition models.

[0170] refer to Figure 6 As shown, in the case where a target sub-recognition model is sequentially determined among multiple sub-recognition models, and there is a next sub-recognition model to be determined after the current target sub-recognition model has been determined, the image recognition apparatus of the exemplary embodiments of this disclosure may include:

[0171] The first image recognition module 610 inputs the image to be recognized into the current target sub-recognition model so that the current target sub-recognition model outputs first image feature information, and performs image recognition based on the first image feature information to obtain the first image recognition result;

[0172] The first decision module 620 uses the decision-maker between the current target sub-recognition model and the next sub-recognition model to be determined as the target decision-maker, and inputs the first image feature information and the first image recognition result into the target decision-maker to score the first image recognition result to obtain a score value.

[0173] The determination module 630, if the score value meets the standard, determines the first image recognition result as the final image recognition result and stops determining the next sub-recognition model after the current target sub-recognition model as the target sub-recognition model.

[0174] Optionally, module 530 is further used for:

[0175] If the score does not meet the target, the next sub-recognition model to be determined is the next target sub-recognition model, and the first image recognition module 610 is returned based on the next target sub-recognition model.

[0176] Optionally, the determination module 630 is also specifically used for:

[0177] If the last sub-recognition model in the image recognition model is determined to be the target sub-recognition model, the first image recognition result output by the last sub-recognition model is determined as the final image recognition result.

[0178] Optionally, module 630 is specifically used for:

[0179] If the score is not lower than the first threshold, the score is considered to have met the standard.

[0180] Optionally, the first image recognition result is represented as a probability value;

[0181] The first decision module 620 is specifically used for:

[0182] The target decision unit weights the first image feature information by multiplication based on probability values ​​to obtain the target image features, and predicts the score value of the first image recognition result based on the target image features.

[0183] Optionally, the first decision module 620 is also specifically used for:

[0184] The target decision unit integrates the features of the first image to obtain integrated image features.

[0185] The target image features are obtained by multiplying the integrated image features based on probability values ​​using a target decision unit.

[0186] Optionally, the decision maker includes at least two sets of sequentially connected feature processing units. The at least two sets of sequentially connected feature processing units are divided into a first part and a second part based on the connection relationship. The second part includes only the last set of feature processing units, and the first part includes all feature processing units except the last set of feature processing units. Each set of feature processing units includes a first fully connected layer, a batch normalization layer and a nonlinearization layer connected in sequence.

[0187] The first decision module 620 is also specifically used for:

[0188] The first image feature information is input into the first part, and the first image feature information is integrated sequentially by each feature processing unit in the first part until the integrated image features are obtained.

[0189] The probability value and the integrated image features are input into the second part, so that the final set of feature processing units in the second part can multiply the integrated image features based on the probability value to obtain the target image features.

[0190] Optionally, the decision-maker further includes a second fully connected layer and an activation function connected after the second part; the first decision module 620 is specifically used for:

[0191] The target image features are scored and predicted using a second fully connected layer.

[0192] The predicted values ​​are nonlinearized by an activation function to obtain the score of the first image recognition result.

[0193] Optionally, the network structure of multiple cascaded sub-recognition models becomes increasingly refined according to the cascading relationship from front to back.

[0194] Optionally, the first decision module 620 is specifically used for:

[0195] The first image recognition result is compared with the second threshold;

[0196] If the first image recognition result is greater than the second threshold, then the final image recognition result is determined to be an image of the target class.

[0197] If the first image recognition result is not greater than the second threshold, then the final image recognition result is determined to be an image of the target class not recognized.

[0198] The image recognition apparatus of this embodiment deploys an image recognition model, which includes multiple cascaded sub-recognition models, and adds a decision unit between adjacent sub-recognition models. In this case, the decision unit processes the image feature information extracted by the current target sub-recognition model and the image recognition result output by the current sub-recognition model to score the image recognition result output by the current sub-recognition model. If the score value meets the standard, the current image recognition result is determined as the final image recognition result, and the process of sending the image to be recognized to the next sub-recognition model is stopped.

[0199] In this embodiment, the addition of the decision-maker not only eliminates the cost of manually setting thresholds, but also the decision-making process changes dynamically with the number of predictions, avoiding false detections and missed detections caused by the sensitivity of prediction accuracy to thresholds, thereby improving the accuracy and efficiency of image recognition.

[0200] Since the functional modules of the image recognition device in this embodiment are the same as those in the above-described image recognition method embodiments, they will not be described again here.

[0201] Next, refer to Figure 7A training apparatus for an image recognition model according to an exemplary embodiment of the present disclosure is described. The image recognition model to be trained includes multiple cascaded sub-recognition models and a decision unit to be trained connected between adjacent sub-recognition models; the training apparatus for the image recognition model includes the following modules for training each decision unit to be trained:

[0202] refer to Figure 7 As shown, the training apparatus for the image recognition model in an exemplary embodiment of this disclosure may include:

[0203] The second image recognition module 710, for the current decision-maker to be trained, inputs the sample image into the two adjacent sub-recognition models before and after the current decision-maker to be trained, so that the sub-recognition model before the current decision-maker to be trained outputs the second image feature information and predicts the second image recognition result based on the second image feature information, and the sub-recognition model after the current decision-maker to be trained outputs the third image recognition result.

[0204] The second decision module 720 inputs the second image feature information and the second image recognition result into the current decision-maker to be trained, so as to score the second image recognition result through the current decision-maker to be trained to obtain a score value.

[0205] The training module 730 performs a correctness check on the second image recognition result based on the real result of the sample image to obtain a first check result, and performs a correctness check on the third image recognition result based on the real result to obtain a second check result. It also obtains a reward value for the score value based on the consistency relationship between the first check result and the second check result, calculates a loss function based on the score value and the reward value, and uses the loss function to train the current decision-maker to be trained.

[0206] Optionally, the training module 730 is also specifically used for:

[0207] Before inputting the sample image into the current decision-maker to be trained, and before the two adjacent sub-recognition models, the two adjacent sub-recognition models before and after the current decision-maker to be trained are trained to obtain the two trained sub-recognition models.

[0208] Optionally, the training module 730 is specifically used for:

[0209] By using the first and second test results to look up a table, the consistency relationship between the first and second test results and the reward value matched by the consistency relationship can be obtained.

[0210] Optionally, the second image recognition result is represented as a probability value;

[0211] The second decision module 720 is specifically used for:

[0212] The current decision maker to be trained calculates the target image features by multiplying the second image feature information based on probability values, and then predicts the score of the second image recognition result based on the target image features.

[0213] Optionally, the second decision module 720 is also specifically used for:

[0214] The second image feature information is integrated by the decision-maker to be trained to obtain integrated image features;

[0215] The target image features are obtained by multiplying the integrated image features based on probability values ​​using the decision-maker to be trained.

[0216] Optionally, the decision maker includes at least two sets of sequentially connected feature processing units. The at least two sets of sequentially connected feature processing units are divided into a first part and a second part based on the connection relationship. The second part includes only the last set of feature processing units, and the first part includes all feature processing units except the last set of feature processing units. Each set of feature processing units includes a first fully connected layer, a batch normalization layer and a nonlinearization layer connected in sequence.

[0217] The second decision module 720 is also specifically used for:

[0218] The second image feature information is input into the first part, and the feature processing units in the first part sequentially integrate the second image feature information until the integrated image features are obtained.

[0219] The probability value and the integrated image features are input into the second part, so that the final set of feature processing units in the second part can multiply the integrated image features based on the probability value to obtain the target image features.

[0220] Optionally, the decision-maker further includes a second fully connected layer and an activation function connected after the second part; the second decision module 720 is specifically used for:

[0221] The target image features are scored and predicted using a second fully connected layer.

[0222] The predicted values ​​are nonlinearized by an activation function to obtain the score of the second image recognition result.

[0223] Optionally, based on the cascading order of the sub-recognition models from front to back, the sub-recognition models in the image recognition model are cascaded sequentially from coarse to fine according to the network structure.

[0224] The training apparatus for the image recognition model in this embodiment replaces the threshold by inserting a decision agent after each cascaded sub-recognition model. The decision agent is trained using reinforcement learning, and a right-or-wrong reward mechanism is designed for the reinforcement learning training, so that an effective decision agent can be stably trained and learned simply by judging whether the predictions of the preceding and following sub-recognition models are correct.

[0225] Once the decision-maker has been trained, it can autonomously decide whether a subsequent sub-recognition model needs to continue prediction based on the image feature information and image recognition results output by the previous sub-recognition model. The addition of the decision-maker not only eliminates the cost of manually setting thresholds, but its decision-making process also dynamically changes with the number of predictions, avoiding false positives and false negatives caused by the sensitivity of prediction accuracy to thresholds, thereby improving image recognition accuracy and efficiency.

[0226] Exemplary storage media

[0227] After introducing the image recognition method and image recognition model training method and apparatus according to exemplary embodiments of the present disclosure, the following will refer to... Figure 7 The storage medium of the exemplary embodiments of this disclosure will be described.

[0228] refer to Figure 8 As shown, a program product 800 for implementing the above-described method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a device such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0229] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0230] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0231] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0232] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0233] Exemplary electronic devices

[0234] Having described the storage medium of exemplary embodiments of this disclosure, the following references are made. Figure 9 An electronic device according to an exemplary embodiment of the present disclosure will be described.

[0235] Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0236] like Figure 9 As shown, the electronic device 900 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, a bus 930 connecting different system components (including storage unit 920 and processing unit 910), and a display unit 940.

[0237] The storage unit stores program code, which can be executed by the processing unit 910 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 910 can perform, as follows: Figure 1 Or the steps shown in 5.

[0238] Storage unit 920 may include volatile storage units, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0239] The storage unit 920 may also include a program / utility 924 having a set (at least one) of program modules 925, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0240] Bus 930 may include a data bus, an address bus, and a control bus.

[0241] Electronic device 900 can also communicate with one or more external devices 901 (e.g., keyboard, pointing device, Bluetooth device, etc.) via input / output (I / O) interface 950. Electronic device 900 also includes a display unit 940 connected to input / output (I / O) interface 950 for display purposes. Furthermore, electronic device 900 can communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 1060. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0242] It should be noted that although several modules or sub-modules of the image recognition device and the training device for the image recognition model have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0243] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0244] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. An image recognition method based on an image recognition model, characterized in that, The image recognition model includes multiple cascaded sub-recognition models and a decision unit connected between two adjacent sub-recognition models; The image recognition method includes: In the case where a target sub-identification model is sequentially determined among multiple sub-identification models, and there is a next sub-identification model to be determined after the current target sub-identification model, the following steps are performed: Recognition steps: Input the image to be recognized into the current target sub-recognition model, so that the current target sub-recognition model outputs first image feature information, and perform image recognition based on the first image feature information to obtain the first image recognition result; Decision-making steps: The decision-maker between the current target sub-recognition model and the next sub-recognition model to be determined is used as the target decision-maker. The first image feature information and the first image recognition result are input into the target decision-maker to score the first image recognition result and obtain a score value. If the score meets the standard, the first image recognition result is determined as the final image recognition result, and the determination of the next sub-recognition model after the current target sub-recognition model as the target sub-recognition model is stopped. The first image recognition result is represented as a probability value; the decision-maker includes at least two sets of sequentially connected feature processing units, which are divided into a first part and a second part based on their connection relationships. The second part includes only the last set of feature processing units, and the first part includes all feature processing units except the last set. Each set of feature processing units includes a first fully connected layer, a batch normalization layer, and a nonlinearization layer connected in sequence; the step of scoring the first image recognition result through the target decision-maker to obtain a score value includes: The first image feature information is input into the first part, and the first image feature information is sequentially integrated by each feature processing unit in the first part until integrated image features are obtained. The probability value and the integrated image features are input into the second part, so that the last group of feature processing units in the second part can multiply the integrated image features based on the probability value to obtain the target image features, and predict the score value of the first image recognition result based on the target image features.

2. The image recognition method based on an image recognition model according to claim 1, characterized in that, The image recognition method further includes: If the score does not meet the target, the undetermined next sub-identification model is determined as the next target sub-identification model, and the identification and decision steps are performed based on the next target sub-identification model.

3. The image recognition method based on an image recognition model according to claim 2, characterized in that, The image recognition method further includes: If the last sub-recognition model in the image recognition model is determined to be the target sub-recognition model, the first image recognition result output by the last sub-recognition model is determined as the final image recognition result.

4. The image recognition method based on an image recognition model according to claim 1, characterized in that, If the score is not lower than the first threshold, the score is determined to meet the standard.

5. The image recognition method based on an image recognition model according to claim 1, characterized in that, The decision maker further includes a second fully connected layer and an activation function connected after the second part; the scoring of predicting the first image recognition result based on the target image features includes: The target image features are scored and predicted using the second fully connected layer. The predicted value is nonlinearized by the activation function to obtain the score value of the first image recognition result.

6. The image recognition method based on an image recognition model according to claim 1, characterized in that, in, According to the cascading relationship from front to back, the network structure of the multiple cascaded sub-identification models becomes increasingly refined.

7. The image recognition method based on an image recognition model according to claim 1, characterized in that, Determining the first image recognition result as the final image recognition result includes: The first image recognition result is compared with the second threshold; If the first image recognition result is greater than the second threshold, then the final image recognition result is determined to be an image of the target class. If the first image recognition result is not greater than the second threshold, then the final image recognition result is determined to be an image of the target class not recognized.

8. A training method for an image recognition model, characterized in that, The image recognition model to be trained includes multiple cascaded sub-recognition models and a decision unit to be trained connected between two adjacent sub-recognition models; the training method of the image recognition model adopts the following steps to train each decision unit to be trained: For the current decision maker to be trained, the sample image is input into the two adjacent sub-recognition models before and after the current decision maker to be trained, so that the sub-recognition model before the current decision maker to be trained outputs the second image feature information and predicts the second image recognition result based on the second image feature information, and the sub-recognition model after the current decision maker to be trained outputs the third image recognition result. The second image feature information and the second image recognition result are input into the current decision-making unit to be trained, so that the second image recognition result is scored by the current decision-making unit to obtain a score value. The correctness of the second image recognition result is checked based on the real result of the sample image to obtain a first check result, and the correctness of the third image recognition result is checked based on the real result to obtain a second check result. A reward value for rewarding the score value is obtained according to the consistency relationship between the first check result and the second check result. A loss function is calculated according to the score value and the reward value, and the current decision-maker to be trained is trained using the loss function.

9. The training method for the image recognition model according to claim 8, characterized in that, The training method for the image recognition model also includes: Before inputting the sample image into the current decision-maker to be trained, and before the two adjacent sub-recognition models, the two adjacent sub-recognition models before and after the current decision-maker to be trained are trained to obtain the two trained sub-recognition models.

10. The training method for the image recognition model according to claim 8, characterized in that, The reward value for awarding the score is obtained based on the consistency relationship between the first test result and the second test result, including: By using the first test result and the second test result to look up a table, the consistency relationship between the first test result and the second test result, as well as the reward value matched by the consistency relationship, can be obtained.

11. The training method for the image recognition model according to claim 8, characterized in that, The second image recognition result is represented by a probability value; The second image recognition result is scored using the current decision-maker to obtain a score value, including: The current decision maker to be trained weights the second image feature information by multiplication based on the probability value to obtain the target image features, and predicts the score value of the second image recognition result based on the target image features.

12. The training method for the image recognition model according to claim 11, characterized in that, The current decision-maker to be trained further includes multiplying the second image feature information based on the probability value to obtain the target image features, and also includes: The second image feature information is integrated using the decision-maker to be trained to obtain integrated image features; The target image features are obtained by multiplying the integrated image features based on the probability values ​​using the decision-maker to be trained.

13. The training method for the image recognition model according to claim 12, characterized in that, The decision maker includes at least two sets of sequentially connected feature processing units. The at least two sets of sequentially connected feature processing units are divided into a first part and a second part based on the connection relationship. The second part includes only the last set of feature processing units. The first part includes all feature processing units except the last set of feature processing units. Each set of feature processing units includes a first fully connected layer, a batch normalization layer and a nonlinearization layer connected in sequence. The second image feature information is integrated using the current decision-maker to obtain integrated image features, including: The second image feature information is input into the first part, and the second image feature information is sequentially integrated by each feature processing unit in the first part until integrated image features are obtained. The target image features are obtained by multiplying the integrated image features based on the probability values ​​using the current decision-maker to train the system, including: The probability value and the integrated image features are input into the second part, so that the final feature processing unit of the second part can perform a weighted multiplication of the integrated image features based on the probability value to obtain the target image features.

14. The training method for the image recognition model according to claim 13, characterized in that, The decision maker further includes a second fully connected layer and an activation function connected after the second part; the prediction of the score value of the second image recognition result based on the target image features includes: The target image features are scored and predicted using the second fully connected layer. The predicted value is nonlinearized by the activation function to obtain the score value of the second image recognition result.

15. The training method for the image recognition model according to claim 8, characterized in that, in, Based on the cascading order of the sub-recognition models from front to back, each sub-recognition model in the image recognition model is cascaded in order from coarse to fine according to the network structure.

16. An image recognition device based on an image recognition model, characterized in that, The image recognition model includes multiple cascaded sub-recognition models and a decision unit connected between two adjacent sub-recognition models; In a scenario where a target sub-recognition model is sequentially determined from among multiple sub-recognition models, and a next sub-recognition model is yet to be determined after the current target sub-recognition model, the image recognition device includes the following modules: The first image recognition module inputs the image to be recognized into the current target sub-recognition model, so that the current target sub-recognition model outputs first image feature information, and performs image recognition based on the first image feature information to obtain a first image recognition result; The first decision module uses the decision-maker between the current target sub-recognition model and the next sub-recognition model to be determined as the target decision-maker, and inputs the first image feature information and the first image recognition result into the target decision-maker to score the first image recognition result to obtain a score value. The determination module, if the score value meets the standard, determines the first image recognition result as the final image recognition result and stops determining the next sub-recognition model after the current target sub-recognition model as the target sub-recognition model; The first image recognition result is represented as a probability value; the decision-maker includes at least two sets of sequentially connected feature processing units, which are divided into a first part and a second part based on their connection relationships. The second part includes only the last set of feature processing units, and the first part includes all feature processing units except the last set. Each set of feature processing units includes a first fully connected layer, a batch normalization layer, and a nonlinearization layer connected in sequence; the first decision module is specifically used for: The first image feature information is input into the first part, and the first image feature information is sequentially integrated by each feature processing unit in the first part until integrated image features are obtained. The probability value and the integrated image features are input into the second part, so that the last group of feature processing units in the second part can multiply the integrated image features based on the probability value to obtain the target image features, and predict the score value of the first image recognition result based on the target image features.

17. The image recognition device based on the image recognition model according to claim 16, characterized in that, The determining module is further specifically used for: If the score does not meet the standard, the pending next sub-recognition model is determined as the next target sub-recognition model, and the first image recognition module is returned based on the next target sub-recognition model.

18. The image recognition device based on the image recognition model according to claim 17, characterized in that, The determining module is further specifically used for: If the last sub-recognition model in the image recognition model is determined to be the target sub-recognition model, the first image recognition result output by the last sub-recognition model is determined as the final image recognition result.

19. The image recognition device based on the image recognition model according to claim 16, characterized in that, The determining module is specifically used for: If the score is not lower than the first threshold, the score is determined to meet the standard.

20. The image recognition device based on an image recognition model according to claim 16, characterized in that, The decision-maker further includes a second fully connected layer and an activation function connected after the second part; the first decision module is specifically used for: The target image features are scored and predicted using the second fully connected layer. The predicted value is nonlinearized by the activation function to obtain the score value of the first image recognition result.

21. The image recognition device based on an image recognition model according to claim 16, characterized in that, in, According to the cascading relationship from front to back, the network structure of the multiple cascaded sub-identification models becomes increasingly refined.

22. The image recognition device based on an image recognition model according to claim 16, characterized in that, The first decision module is specifically used for: The first image recognition result is compared with the second threshold; If the first image recognition result is greater than the second threshold, then the final image recognition result is determined to be an image of the target class. If the first image recognition result is not greater than the second threshold, then the final image recognition result is determined to be an image of the target class not recognized.

23. A training device for an image recognition model, characterized in that, The image recognition model to be trained includes multiple cascaded sub-recognition models and a decision unit to be trained connected between two adjacent sub-recognition models; the training device for the image recognition model includes the following modules for training each decision unit to be trained: The second image recognition module, for the current decision maker to be trained, inputs the sample image into the two adjacent sub-recognition models before and after the current decision maker to be trained, so that the sub-recognition model before the current decision maker to be trained outputs the second image feature information and predicts the second image recognition result based on the second image feature information, and the sub-recognition model after the current decision maker to be trained outputs the third image recognition result. The second decision module inputs the second image feature information and the second image recognition result into the current decision-maker to be trained, so as to score the second image recognition result through the current decision-maker to be trained to obtain a score value. The training module performs a correctness check on the second image recognition result based on the real result of the sample image to obtain a first check result, and performs a correctness check on the third image recognition result based on the real result to obtain a second check result. It also obtains a reward value for the score value based on the consistency relationship between the first check result and the second check result, calculates a loss function based on the score value and the reward value, and uses the loss function to train the current decision-maker to be trained.

24. The training apparatus for the image recognition model according to claim 23, characterized in that, The training module is also specifically used for: Before inputting the sample image into the current decision-maker to be trained, and before the two adjacent sub-recognition models, the two adjacent sub-recognition models before and after the current decision-maker to be trained are trained to obtain the two trained sub-recognition models.

25. The training apparatus for the image recognition model according to claim 23, characterized in that, The training module is specifically used for: By using the first test result and the second test result to look up a table, the consistency relationship between the first test result and the second test result, as well as the reward value matched by the consistency relationship, can be obtained.

26. The training apparatus for the image recognition model according to claim 23, characterized in that, The second image recognition result is represented by a probability value; The second decision module is specifically used for: The current decision maker to be trained weights the second image feature information by multiplication based on the probability value to obtain the target image features, and predicts the score value of the second image recognition result based on the target image features.

27. The training apparatus for the image recognition model according to claim 26, characterized in that, The second decision module is also specifically used for: The second image feature information is integrated using the decision-maker to be trained to obtain integrated image features; The target image features are obtained by multiplying the integrated image features based on the probability values ​​using the decision-maker to be trained.

28. The training apparatus for the image recognition model according to claim 27, characterized in that, The decision maker includes at least two sets of sequentially connected feature processing units. The at least two sets of sequentially connected feature processing units are divided into a first part and a second part based on the connection relationship. The second part includes only the last set of feature processing units. The first part includes all feature processing units except the last set of feature processing units. Each set of feature processing units includes a first fully connected layer, a batch normalization layer and a nonlinearization layer connected in sequence. The second decision module is also specifically used for: The second image feature information is input into the first part, and the second image feature information is sequentially integrated by each feature processing unit in the first part until integrated image features are obtained. The probability value and the integrated image features are input into the second part, so that the final feature processing unit of the second part can perform a weighted multiplication of the integrated image features based on the probability value to obtain the target image features.

29. The training apparatus for the image recognition model according to claim 28, characterized in that, The decision-maker further includes a second fully connected layer and an activation function connected after the second part; the second decision module is specifically used for: The target image features are scored and predicted using the second fully connected layer. The predicted value is nonlinearized by the activation function to obtain the score value of the second image recognition result.

30. The training apparatus for the image recognition model according to claim 23, characterized in that, in, Based on the cascading order of the sub-recognition models from front to back, each sub-recognition model in the image recognition model is cascaded in order from coarse to fine according to the network structure.

31. A storage medium having a computer program stored thereon, characterized in that, The computer program is executed by the processor to achieve the following: The image recognition method based on the image recognition model according to any one of claims 1 to 7; or The training method for the image recognition model according to any one of claims 8 to 15.

32. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the executable instructions: The image recognition method based on the image recognition model according to any one of claims 1 to 7; or The training method for the image recognition model according to any one of claims 8 to 15.

Images