Image recognition method and recognition system based on deep learning
Through multi-scale feature extraction and dynamic probability threshold prediction methods, the problem of low image recognition accuracy in the prior art is solved, and high-precision recognition under complex background and lighting changes is achieved.
Patent Information
- Application Number
- CN202411046361.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-01
AI Technical Summary
The existing CNN-based image recognition method has low recognition accuracy when facing complex backgrounds and lighting changes, and it is difficult to adapt to different lighting conditions for a fixed threshold, which ignores differences in pixel importance, resulting in misidentification or missed recognition.
Using multi-scale feature extraction, pixel-level classification, feature-probability vector splicing and dynamic probability threshold prediction, multi-scale features are extracted through convolutional neural networks, the Softmax activation function is converted into category probability values, and the features are spliced in the feature space- Probability vectors are used to generate dynamic probability thresholds for identification using pre-trained threshold prediction networks.
It realizes more accurate target object recognition, improves the flexibility and accuracy of recognition, adapts to different lighting conditions, reduces misidentification and misidentification, and improves the robustness and efficiency of the identification system.
Smart Images

Figure CN118570563B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to an image recognition method and recognition system based on deep learning. Background Art
[0002] Image recognition is one of the core tasks of computer vision. Traditional image recognition methods mainly rely on manually designed features and machine learning algorithms, such as SIFT and SURF, but the recognition accuracy is often greatly reduced when faced with complex backgrounds, lighting changes, occlusions, etc. In recent years, with the development of deep learning technology, especially the widespread application of convolutional neural networks (CNNs), image recognition technology has made significant progress, and can automatically learn the deep features of images and effectively improve recognition accuracy.
[0003] However, most existing CNN-based image recognition methods use fixed thresholds to classify pixels, which limits the flexibility and accuracy of recognition to a certain extent. For example, under different lighting conditions, the feature expression of the same object may vary greatly, and fixed thresholds are difficult to adapt to such changes, resulting in misidentification or missed recognition.
[0004] In addition, the importance of each pixel in the image (i.e., its contribution to the final recognition result) is not consistent. The fixed threshold ignores this fact, which may cause key information to be ignored and non-key areas to be over-emphasized.
[0005] Therefore, it is necessary to provide an image recognition method and recognition system based on deep learning to solve the above technical problems. Summary of the invention
[0006] To solve the above technical problems, the present invention provides an image recognition method and recognition system based on deep learning. Through multi-scale feature extraction, pixel-level classification, feature-probability vector splicing and dynamic probability threshold prediction, it can not only capture the multi-scale features of the target object in the image, but also dynamically adjust the classification threshold according to the pixel-level information, thereby achieving more accurate target recognition.
[0007] The present invention provides an image recognition method based on deep learning, and the recognition method comprises the following steps:
[0008] Extracting a multi-scale feature vector of the image to be identified using a convolutional neural network, wherein the multi-scale feature vector includes a multi-scale feature element corresponding to each pixel in the image to be identified;
[0009] Traversing each pixel in the image to be identified, and using a Softmax activation function to convert the multi-scale feature elements of the pixel into a category probability value;
[0010] Mapping the multi-scale feature elements of each pixel to a feature space, and concatenating the multi-scale feature elements of each pixel with the corresponding category probability value in the feature space to obtain a feature-probability vector for each pixel;
[0011] Input the feature-probability vector of each pixel into the pre-trained threshold prediction network one by one for prediction, and output the dynamic probability threshold;
[0012] By comparing the category probability value of each pixel with the dynamic probability threshold, pixels with a probability value higher than the dynamic probability threshold are assigned to the category of the target object, and the recognition result corresponding to the target object is obtained.
[0013] Preferably, the extracting a multi-scale feature vector of the image to be identified by using a convolutional neural network includes:
[0014] Build a convolutional neural network based on the U-Net architecture;
[0015] Performing image preprocessing on the image to be identified, wherein the image preprocessing includes resizing and normalization;
[0016] Extracting features of different scales of the preprocessed image to be identified through a convolutional layer with convolutional kernels of different sizes in a convolutional neural network based on a U-Net architecture;
[0017] The features of different scales are merged by upsampling to obtain a multi-scale feature vector of the image to be identified;
[0018] The multi-scale feature vector of the image to be identified is segmented at the pixel level to obtain a multi-scale feature element for each pixel in the image to be identified.
[0019] Preferably, traversing each pixel in the image to be identified and converting the multi-scale feature elements of the pixel into a category probability value using a Softmax activation function includes:
[0020] Flatten the multi-scale feature elements of each pixel to obtain a one-dimensional vector for each pixel;
[0021] The one-dimensional vector of each pixel is mapped to the category space through the fully connected layer to obtain the category vector of each pixel;
[0022] The Softmax function is used to convert the category vector of each pixel into a category probability value.
[0023] Preferably, mapping the multi-scale feature element of each pixel to a feature space, and concatenating the multi-scale feature element of each pixel with a corresponding category probability value in the feature space, comprises:
[0024] Mapping the multi-scale feature element of each pixel to a predefined high-dimensional feature space through a nonlinear transformation, so as to represent the multi-scale feature element of each pixel as a specific vector in the high-dimensional feature space;
[0025] In the high-dimensional feature space, the specific vector of each pixel and the category vector of the corresponding category probability value are concatenated in series to form a feature-probability vector including the specific vector and the category vector.
[0026] Preferably, the step of constructing the threshold prediction network includes:
[0027] Collect a data set including sample feature-probability vectors and corresponding sample threshold labels, and divide it into a training set and a validation set;
[0028] Optimizing the network weights of a threshold prediction network based on a neural network by using the training set through a back propagation algorithm until the threshold prediction network converges;
[0029] The threshold prediction network is evaluated using the validation set, and parameters of the threshold prediction network are adjusted according to the evaluation result to obtain a trained threshold prediction network.
[0030] Preferably, the step of inputting the feature-probability vector of each pixel into a pre-trained threshold prediction network one by one for prediction and outputting a dynamic probability threshold comprises:
[0031] Input the feature-probability vector of each pixel into the pre-trained threshold prediction network in turn;
[0032] The input feature-probability vector is forward propagated through the threshold prediction network, and a dynamic probability threshold is finally output.
[0033] Preferably, the step of comparing the category probability value of each pixel with a dynamic probability threshold, assigning pixels above the dynamic probability threshold to the category of the target object, and obtaining the recognition result corresponding to the target object includes:
[0034] Traversing each pixel in the image to be identified, comparing the category probability value of each pixel with the dynamic probability threshold, if the category probability value is greater than or equal to the dynamic probability threshold, the pixel is considered to belong to the category of the target object; otherwise, the pixel is considered not to belong to the category of the target object;
[0035] According to the comparison results, pixels above the dynamic probability threshold are assigned to the category of the target object;
[0036] The allocation results of all pixels are summarized to obtain the target object recognition result of the image to be recognized.
[0037] The present invention also provides an image recognition system based on deep learning, which is used to perform an image recognition method based on deep learning. The recognition system includes:
[0038] A multi-scale feature extraction module, used to extract a multi-scale feature vector of the image to be identified by using a convolutional neural network, wherein the multi-scale feature vector includes a multi-scale feature element corresponding to each pixel in the image to be identified;
[0039] A probability prediction module is used to traverse each pixel in the image to be identified and convert the multi-scale feature elements of the pixel into a category probability value using a Softmax activation function;
[0040] A feature-probability fusion module is used to map the multi-scale feature elements of each pixel to the feature space, and to concatenate the multi-scale feature elements of each pixel with the corresponding category probability value in the feature space to obtain a feature-probability vector for each pixel;
[0041] The dynamic threshold prediction module is used to input the feature-probability vector of each pixel into the pre-trained threshold prediction network one by one for prediction and output the dynamic probability threshold;
[0042] The result generation module is used to compare the category probability value of each pixel with the dynamic probability threshold, assign the pixels above the dynamic probability threshold to the category of the target object, and obtain the recognition result corresponding to the target object.
[0043] Compared with the related art, the image recognition method and recognition system based on deep learning provided by the present invention have the following beneficial effects:
[0044] The present invention utilizes a convolutional neural network to extract the multi-scale feature vector of the image to be identified, and converts the multi-scale feature elements of pixels into category probability values through the Softmax activation function. Subsequently, the feature elements and the category probability values are concatenated in the feature space to form a feature-probability vector. Finally, a dynamic probability threshold is generated through a pre-trained threshold prediction network and compared with the category probability value to achieve accurate identification of the target object. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A flowchart of an image recognition method based on deep learning provided by the present invention;
[0046] Figure 2 A module structure diagram of an image recognition system based on deep learning provided by the present invention. DETAILED DESCRIPTION
[0047] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.
[0048] It should also be noted that, for ease of description, only the parts related to the present invention, but not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as being processed sequentially, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the various operations can be rearranged. The processing can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc. Embodiment 1
[0049] The present invention provides an image recognition method based on deep learning, referring to Figure 1 As shown, the identification method includes the following steps:
[0050] S1: Extracting a multi-scale feature vector of an image to be identified using a convolutional neural network, wherein the multi-scale feature vector includes a multi-scale feature element corresponding to each pixel in the image to be identified.
[0051] In this embodiment, features of different scales are extracted by adjusting different convolutional layers of the convolutional neural network. For example, shallower convolutional layers can capture local features such as edges and textures in the image, while deeper convolutional layers can learn more abstract and complex high-level features, such as partial or overall structures of objects, to ensure a multi-level and multi-angle understanding of the target objects in the image to be identified; at the same time, the output feature maps of different convolutional layers are converted into multi-scale feature vectors of fixed length.
[0052] S2: traverse each pixel in the image to be identified, and use the Softmax activation function to convert the multi-scale feature elements of the pixel into category probability values.
[0053] In this embodiment, the Softmax function maps the multi-scale feature elements to a probability distribution, and the output value of each category represents the probability that the pixel belongs to that category, and the sum of the probability values of all categories is 1; through the Softmax function, each pixel point will obtain a category probability vector, and each element in the vector represents the possibility that the pixel belongs to that category. This not only realizes the conversion from feature space to probability space, but also provides the necessary probability information for subsequent dynamic probability threshold prediction, thereby enhancing the interpretability and accuracy of recognition decisions.
[0054] S3: Mapping the multi-scale feature element of each pixel to the feature space, and concatenating the multi-scale feature element of each pixel with the corresponding category probability value in the feature space to obtain a feature-probability vector of each pixel.
[0055] In this embodiment, the obtained multi-scale feature elements of each pixel are spliced together with the corresponding category probability vector to form a composite vector containing feature information and probability information, namely, a feature-probability vector. The construction of the feature-probability vector integrates pixel-level feature representation and probability estimation, providing more comprehensive information support for subsequent dynamic probability threshold prediction, which helps the model make more accurate decisions during the recognition process.
[0056] S4: Input the feature-probability vector of each pixel into the pre-trained threshold prediction network one by one for prediction, and output the dynamic probability threshold.
[0057] In this embodiment, a threshold prediction network that has been pre-trained for predicting dynamic probability thresholds is used. Based on the input of the feature-probability vector, the threshold prediction network can predict the dynamic probability threshold. The introduction of the dynamic probability threshold enables the model to flexibly adjust the classification criteria according to the image content and pixel characteristics, avoiding the limitations brought by fixed thresholds and significantly improving the flexibility and accuracy of recognition.
[0058] S5: By comparing the category probability value of each pixel with the dynamic probability threshold, the pixels with a probability value higher than the dynamic probability threshold are assigned to the category of the target object, and the recognition result corresponding to the target object is obtained.
[0059] In this embodiment, the category probability value of each pixel obtained in step S2 is compared with the overall dynamic probability threshold. If the category probability value is equal to or greater than the dynamic probability threshold, the pixel is considered to belong to the target object; otherwise, it is classified as background or non-target part; and based on the above classification process, an image with the target object position and category marked is finally generated, that is, the recognition result.
[0060] The application of dynamic probability threshold enables the recognition algorithm to reduce unnecessary computational overhead while maintaining a high accuracy rate, especially when processing large-scale image data sets. In addition, since the threshold is dynamically adjusted according to the characteristics of the entire image, the recognition system can show better adaptability and robustness when facing different scenarios.
[0061] Specifically, step S1 specifically includes:
[0062] Step S101: Construct a convolutional neural network based on the U-Net architecture.
[0063] In this embodiment, a deep convolutional neural network (CNN) based on the U-Net architecture is constructed. The network consists of an encoder (downsampling path) and a decoder (upsampling path). The encoder is responsible for capturing the detailed features in the image, while the decoder focuses on restoring the spatial details of the image. The combination of the two enables the network to effectively extract and utilize the multi-scale features of the image.
[0064] Step S102: performing image preprocessing on the image to be recognized, wherein the image preprocessing includes resizing and normalization.
[0065] In this embodiment, the input image to be recognized is preprocessed, mainly including two steps of resizing and normalization. Resizing ensures that the image size meets the network input requirements, while normalization scales the pixel values to a specific range (for example, between 0 and 1) to improve the training convergence speed and performance of the network.
[0066] Step S103: extracting features of different scales of the preprocessed image to be identified through a convolutional layer with convolutional kernels of different sizes in a convolutional neural network based on a U-Net architecture.
[0067] In this embodiment, the encoder part consists of a series of convolutional layers, each layer usually contains two 3x3 convolution kernels, followed by a maximum pooling layer for downsampling. This series of convolution and downsampling operations gradually reduces the size of the feature map, while the number of feature channels gradually increases, thereby capturing higher-level abstract features. In each layer, by stacking convolution kernels of the same size, the network is able to learn local features at a specific scale. In addition, at the end of each layer of the encoder, the current feature map is saved so that it can be used by the subsequent decoder.
[0068] The decoder part is the reverse operation of the encoder process, which gradually restores the size of the feature map through upsampling and convolution layers while reducing the number of feature channels. At each decoder layer, an upsampling operation is first performed, and then the result is concatenated with the feature map of the corresponding layer in the encoder to introduce multi-scale information. After that, two 3x3 convolution operations are performed to further fuse these features and restore details. This jump connection mechanism allows the decoder to utilize low-level features from the encoder to restore the details of the image while utilizing high-level features to maintain semantic information.
[0069] Step S104: fusing features of different scales by upsampling to obtain a multi-scale feature vector of the image to be identified.
[0070] In this embodiment, the decoder part of U-Net fuses the different scale features extracted by the encoder stage through upsampling operation. While upsampling, the network will splice the features of the decoder with the corresponding layer features of the encoder to restore more detailed information, and finally form a multi-scale feature vector that integrates multiple scale features.
[0071] Step S105: performing pixel-level segmentation on the multi-scale feature vector of the image to be identified to obtain a multi-scale feature element for each pixel in the image to be identified.
[0072] In this embodiment, the fused multi-scale feature vector is converted into a feature map of the same size as the original image through the last output layer of U-Net, and then pixel-level segmentation is performed to obtain multi-scale feature elements of each pixel. This process ensures that the feature information of each pixel is fully retained, providing a basis for subsequent classification and recognition.
[0073] Specifically, step S2 specifically includes:
[0074] Step S201: flatten the multi-scale feature element of each pixel to obtain a one-dimensional vector of each pixel.
[0075] In this embodiment, in order to make the multi-scale feature vector suitable for the input of the fully connected layer, the multi-scale feature elements of each pixel are flattened, which means that the feature map of each pixel is converted from a two-dimensional matrix to a one-dimensional vector. The purpose of this is to convert the multi-scale feature elements into a form that can be processed by the fully connected layer for the next step of category mapping. The flattening process not only simplifies the data structure, but also ensures that the feature information of each pixel is completely retained during the conversion process.
[0076] Step S202: Map the one-dimensional vector of each pixel to the category space through a fully connected layer to obtain a category vector for each pixel.
[0077] In this embodiment, the flattened one-dimensional vector of each pixel is input into the fully connected layer, which maps the feature vector of each pixel to a higher-dimensional space, where each dimension corresponds to a possible category. By learning the weight matrix and bias term, the fully connected layer can convert the feature vector into a category vector, where each element represents the unnormalized score (logits) of the pixel belonging to a specific category. This process essentially converts the multi-scale feature elements of each pixel into a preliminary estimate of the possibility of each category, laying the foundation for subsequent probability conversion.
[0078] Step S203: Use the Softmax function to convert the category vector of each pixel into a category probability value.
[0079] In this embodiment, the Softmax function is applied to the category vector output by the fully connected layer. The Softmax function can convert the category vector of each pixel into a probability distribution, in which each value represents the probability that the pixel belongs to a specific category. The Softmax function converts the original unnormalized score into a probability value, ensuring that the probability values of all categories add up to 1, thus forming a reasonable probability distribution. This step is crucial to the decision-making process of the model because it not only completes the conversion from feature space to probability space, but also provides probability information for subsequent dynamic probability threshold prediction, enhancing the interpretability and accuracy of the recognition results.
[0080] Specifically, step S3 specifically includes:
[0081] Step S301: Mapping the multi-scale feature element of each pixel to a predefined high-dimensional feature space through a nonlinear transformation, so as to represent the multi-scale feature element of each pixel as a specific vector in the high-dimensional feature space.
[0082] In this embodiment, in order to enhance the expressiveness and distinguishability of features, we map the multi-scale feature elements of each pixel to a high-dimensional feature space through nonlinear transformations, including but not limited to ReLU, tanh and Sigmoid activation functions, which can introduce nonlinearity to the model and help capture more complex feature relationships. In the high-dimensional feature space, the multi-scale feature elements of each pixel are converted into a specific vector, which not only contains the original feature information, but also adds additional representation capabilities through nonlinear transformations, so that the model can better understand and distinguish different image features.
[0083] Step S302: In the high-dimensional feature space, the specific vector of each pixel and the category vector of the corresponding category probability value are concatenated in series to form a feature-probability vector including the specific vector and the category vector.
[0084] In this embodiment, in order to integrate feature representation and probability information, a concatenation method is adopted in the high-dimensional feature space to concatenate the specific vector of each pixel with the category vector of the corresponding category probability value. This means that the specific vector and the category vector are arranged in a certain order to form a new vector, namely, a feature-probability vector. This feature-probability vector construction method not only helps the model to make more accurate decisions during the recognition process, but also improves the adaptability and robustness of the recognition system, especially when processing complex image data sets, and can effectively cope with changing scenes and targets.
[0085] Specifically, in step S4, the step of constructing the threshold prediction network includes:
[0086] A dataset including sample feature-probability vectors and corresponding sample threshold labels is collected and divided into a training set and a validation set.
[0087] In this embodiment, a data set containing sample feature-probability vectors and their corresponding sample threshold labels is prepared. These sample feature-probability vectors are derived from a set of feature-probability vectors obtained in an image recognition task, while the sample threshold labels are derived from domain expert knowledge or historical data statistics to guide the model to learn how to correctly predict dynamic probability thresholds. The data set is then divided into a training set and a validation set, where the training set is used to train the threshold prediction network, and the validation set is used to evaluate the generalization ability and performance of the model to ensure that the model not only performs well on the training data, but also makes accurate predictions on unseen data.
[0088] The training set is used to optimize the network weights of a threshold prediction network based on a neural network through a back propagation algorithm until the threshold prediction network converges.
[0089] In this embodiment, the network weights of the threshold prediction network are optimized by the back propagation algorithm using the data in the training set, wherein the back propagation algorithm is an efficient gradient descent method that can calculate the gradient of the loss function with respect to the network weights and update the weights accordingly to minimize the difference between the predicted threshold and the actual threshold. This process continues iteratively until the network reaches a convergence state, that is, the update amplitude of the network weights becomes very small, indicating that the model has learned an effective threshold prediction strategy.
[0090] The threshold prediction network is evaluated using the validation set, and parameters of the threshold prediction network are adjusted according to the evaluation result to obtain a trained threshold prediction network.
[0091] In this embodiment, a validation set is used to evaluate the performance of the threshold prediction network. Specifically, the prediction error of the model on the validation set is calculated, such as the mean square error (MSE) or the mean absolute error (MAE), and the network parameters are fine-tuned according to the evaluation results. If the model performs poorly on the validation set, adjust the network structure, learning rate, or other hyperparameters to improve the generalization ability of the model. This process may require multiple iterations until a set of optimal parameters is found so that the performance of the threshold prediction network on the validation set is optimal, thereby ensuring that the model can accurately predict the dynamic probability threshold in practical applications and improve the accuracy and efficiency of image recognition.
[0092] Specifically, step S4 specifically includes:
[0093] S401: Input the feature-probability vector of each pixel into the pre-trained threshold prediction network in sequence.
[0094] In this embodiment, the feature-probability vector of each pixel obtained from step S3 is input into the pre-trained threshold prediction network one by one. The pre-trained threshold prediction network is a neural network model specially designed for predicting dynamic probability thresholds. It has been trained on a large number of sample feature-probability vectors and corresponding sample threshold label data sets, thereby learning how to predict the appropriate dynamic probability threshold based on the feature-probability vector.
[0095] S402: forward propagating the input feature-probability vector through the threshold prediction network, and finally outputting a dynamic probability threshold.
[0096] In this embodiment, after the feature-probability vector is input to the threshold prediction network, the network will perform a forward propagation process. In this process, the feature-probability vector will pass through a series of neurons and weight matrices, undergo a nonlinear transformation through an activation function, and ultimately generate a dynamic probability threshold at the output layer of the network. This dynamic probability threshold is determined based on the characteristics of the input feature-probability vector and the pattern learned by the network. It can change with changes in the image content, providing a personalized classification standard for each pixel. This dynamically adjusted threshold can more accurately reflect the true situation of the image, thereby improving the accuracy and robustness of image recognition.
[0097] Specifically, step S5 specifically includes:
[0098] S501: traverse each pixel in the image to be identified, and compare the category probability value of each pixel with the dynamic probability threshold. If the category probability value is greater than or equal to the dynamic probability threshold, the pixel is considered to belong to the category of the target object; otherwise, the pixel is considered not to belong to the category of the target object.
[0099] In this embodiment, each pixel in the entire image to be identified is traversed. For each pixel, the category probability value of the pixel obtained from step S3 (i.e., the probability of belonging to the target category) is compared with the dynamic probability threshold obtained from step S4. If the category probability value of the pixel is greater than or equal to its corresponding dynamic probability threshold, then the pixel is determined to belong to the category of the target; conversely, if the category probability value is less than the dynamic probability threshold, it is considered that the pixel does not belong to the category of the target. This process ensures that only those pixels with sufficiently high probability values are identified as part of the target, thereby avoiding incorrect classification.
[0100] S502: According to the comparison result, pixels with a probability higher than the dynamic probability threshold are assigned to the category of the target object.
[0101] In this embodiment, based on the comparison result in S501, all pixels whose class probability values are greater than or equal to the dynamic probability threshold are marked or assigned as target object categories. These pixels constitute the outline and details of the target in the image, and they will be used to construct the complete shape of the target. This allocation process is performed pixel by pixel, ensuring that the classification of each pixel is based on its own characteristics and its relationship with the dynamic probability threshold.
[0102] S503: Summarize the allocation results of all pixels to obtain the target object recognition result of the image to be recognized.
[0103] In this embodiment, the classification results of all pixels are summarized. All pixels assigned to the target category are collected and integrated to form a coherent area, which represents the target identified in the image. In this way, the precise position, shape, size and other information of the target in the entire image to be identified can be obtained to complete the identification of the target. This result can be further used in subsequent analysis, positioning, tracking or other computer vision tasks to achieve effective management and application of the target.
[0104] The working principle of the image recognition method based on deep learning provided by the present invention is as follows: a multi-scale feature vector of the image to be recognized is extracted by using a convolutional neural network, and the multi-scale feature elements of the pixels are converted into category probability values through a Softmax activation function. Subsequently, the feature elements and the category probability values are spliced in the feature space to form a feature-probability vector. Finally, a dynamic probability threshold is generated through a pre-trained threshold prediction network and compared with the category probability value to achieve accurate recognition of the target object. Embodiment 2
[0105] The present invention also provides an image recognition system based on deep learning, which is used to perform an image recognition method based on deep learning. Figure 2 As shown, the identification system includes:
[0106] The multi-scale feature extraction module 100 is used to extract a multi-scale feature vector of an image to be identified by using a convolutional neural network, wherein the multi-scale feature vector includes a multi-scale feature element corresponding to each pixel in the image to be identified.
[0107] The probability prediction module 200 is used to traverse each pixel in the image to be identified and convert the multi-scale feature elements of the pixel into a category probability value using a Softmax activation function.
[0108] The feature-probability fusion module 300 is used to map the multi-scale feature elements of each pixel to the feature space, and concatenate the multi-scale feature elements of each pixel with the corresponding category probability value in the feature space to obtain the feature-probability vector of each pixel.
[0109] The dynamic threshold prediction module 400 is used to input the feature-probability vector of each pixel into the pre-trained threshold prediction network one by one for prediction, and output a dynamic probability threshold.
[0110] The result generation module 500 is used to compare the category probability value of each pixel with the dynamic probability threshold, assign pixels with a probability value higher than the dynamic probability threshold to the category of the target object, and obtain the recognition result corresponding to the target object.
[0111] The working principle of the deep learning-based image recognition system provided by the present invention is as follows: a convolutional neural network is used in a multi-scale feature extraction module 100 to extract a multi-scale feature vector of an image to be identified, and then, a Softmax activation function is used in a probability prediction module 200 to convert the multi-scale feature elements of pixels into category probability values, and then, in a feature-probability fusion module 300, the feature elements and the category probability values are spliced in the feature space to form a feature-probability vector, and finally, in a dynamic threshold prediction module 400, a dynamic probability threshold is generated by a pre-trained threshold prediction network and compared with the category probability value to achieve accurate recognition of the target object.
[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.
[0113] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically-erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or includes elements that are inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.
Claims
1. An image recognition method based on deep learning, characterized in that: The identification method includes the following steps: Extracting a multi-scale feature vector of the image to be identified using a convolutional neural network, wherein the multi-scale feature vector includes a multi-scale feature element corresponding to each pixel in the image to be identified; Traversing each pixel in the image to be identified, and using a Softmax activation function to convert the multi-scale feature elements of the pixel into a category probability value; Specifically include: Flatten the multi-scale feature elements of each pixel to obtain a one-dimensional vector for each pixel; The one-dimensional vector of each pixel is mapped to the category space through the fully connected layer to obtain the category vector of each pixel; Use the Softmax function to convert the category vector of each pixel into a category probability value; Mapping the multi-scale feature elements of each pixel to a feature space, and concatenating the multi-scale feature elements of each pixel with the corresponding category probability value in the feature space to obtain a feature-probability vector for each pixel; Specifically include: Mapping the multi-scale feature element of each pixel to a predefined high-dimensional feature space through a nonlinear transformation, so as to represent the multi-scale feature element of each pixel as a specific vector in the high-dimensional feature space; In the high-dimensional feature space, the specific vector of each pixel and the category vector of the corresponding category probability value are concatenated in series to form a feature-probability vector including the specific vector and the category vector; Input the feature-probability vector of each pixel into the pre-trained threshold prediction network one by one for prediction, and output the dynamic probability threshold; By comparing the category probability value of each pixel with the dynamic probability threshold, pixels with a probability value higher than the dynamic probability threshold are assigned to the category of the target object, and the recognition result corresponding to the target object is obtained.
2. The image recognition method based on deep learning according to claim 1, characterized in that: The method of extracting a multi-scale feature vector of an image to be identified by using a convolutional neural network includes: Build a convolutional neural network based on the U-Net architecture; Performing image preprocessing on the image to be identified, wherein the image preprocessing includes resizing and normalization; Extracting features of different scales of the preprocessed image to be identified through a convolutional layer with convolutional kernels of different sizes in a convolutional neural network based on a U-Net architecture; The features of different scales are merged by upsampling to obtain a multi-scale feature vector of the image to be identified; The multi-scale feature vector of the image to be identified is segmented at the pixel level to obtain a multi-scale feature element for each pixel in the image to be identified.
3. The image recognition method based on deep learning according to claim 2, characterized in that: The steps of constructing the threshold prediction network include: Collect a data set including sample feature-probability vectors and corresponding sample threshold labels, and divide it into a training set and a validation set; Optimizing the network weights of a threshold prediction network based on a neural network by using the training set through a back propagation algorithm until the threshold prediction network converges; The threshold prediction network is evaluated using the validation set, and parameters of the threshold prediction network are adjusted according to the evaluation result to obtain a trained threshold prediction network.
4. The image recognition method based on deep learning according to claim 3, characterized in that: The feature-probability vector of each pixel is input into the pre-trained threshold prediction network one by one for prediction, and the dynamic probability threshold is output, including: Input the feature-probability vector of each pixel into the pre-trained threshold prediction network in turn; The input feature-probability vector is forward propagated through the threshold prediction network, and a dynamic probability threshold is finally output.
5. The image recognition method based on deep learning according to claim 4, characterized in that: By comparing the category probability value of each pixel with the dynamic probability threshold, pixels with a probability value higher than the dynamic probability threshold are assigned to the category of the target object, and the recognition result corresponding to the target object is obtained, including: Traversing each pixel in the image to be identified, comparing the category probability value of each pixel with the dynamic probability threshold, if the category probability value is greater than or equal to the dynamic probability threshold, the pixel is considered to belong to the category of the target object; otherwise, the pixel is considered not to belong to the category of the target object; According to the comparison results, pixels above the dynamic probability threshold are assigned to the category of the target object; The allocation results of all pixels are summarized to obtain the target object recognition result of the image to be recognized.
6. A deep learning-based image recognition system, executing a deep learning-based image recognition method according to any one of claims 1 to 5, characterized in that: The identification system includes: A multi-scale feature extraction module, used to extract a multi-scale feature vector of the image to be identified by using a convolutional neural network, wherein the multi-scale feature vector includes a multi-scale feature element corresponding to each pixel in the image to be identified; A probability prediction module is used to traverse each pixel in the image to be identified and convert the multi-scale feature elements of the pixel into a category probability value using a Softmax activation function; A feature-probability fusion module is used to map the multi-scale feature elements of each pixel to the feature space, and to concatenate the multi-scale feature elements of each pixel with the corresponding category probability value in the feature space to obtain a feature-probability vector for each pixel; The dynamic threshold prediction module is used to input the feature-probability vector of each pixel into the pre-trained threshold prediction network one by one for prediction and output the dynamic probability threshold; The result generation module is used to compare the category probability value of each pixel with the dynamic probability threshold, assign the pixels above the dynamic probability threshold to the category of the target object, and obtain the recognition result corresponding to the target object.
