Image recognition method, device, equipment and storage medium based on deep learning

By introducing attention mechanisms in deep learning image recognition, optimized model training through feature extraction and loss value calculation, the problems of slow training time and low prediction accuracy are solved, and faster training and higher recognition accuracy are achieved.

CN112767366BActive Publication Date: 2024-11-29NANJING INOVANCE IMAGE VISION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110093285.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-22
Publication Date
2024-11-29
Estimated Expiration
2041-01-22

AI Technical Summary

Technical Problem

When using deep learning for image recognition, the training time is slow and the prediction accuracy is low. It is especially impossible to accurately judge abnormal samples that exceed the label range, and there are high confidence error predictions.

Method used

By receiving training images, obtaining category labels and binary segmentation labels, performing feature extraction and attention prediction, obtaining the target heat map and segmentation loss value, obtaining the embedding vector and category center, calculating the distance loss value and category loss value, and integrating parameter adjustment Convolutional neural networks are used to complete model training and introduce attention mechanisms to improve recognition efficiency and accuracy.

Benefits of technology

This achieves image recognition effect with faster training time and higher prediction accuracy, and optimizes the performance of deep learning models through attention mechanisms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112767366B_ABST
    Figure CN112767366B_ABST
Patent Text Reader

Abstract

The present invention discloses an image recognition method, device, equipment and storage medium based on deep learning. The method comprises: obtaining the category label and binary segmentation label of the training image, extracting the features of the training image, and obtaining the feature space; performing attention prediction on the feature space through a preset convolutional neural network, obtaining the target heat map, and obtaining the segmentation loss value according to the target heat map and the binary segmentation label; obtaining the embedding vector corresponding to the feature space and the category center corresponding to the category label, and calculating the distance loss value between the category center and the embedding vector through a preset loss function; integrating the embedding vector to obtain the image category, calculating the category loss value of the image category and the category label, and adjusting the parameters of the preset convolutional neural network according to the segmentation loss value, the distance loss value and the category loss value to complete the model training. The present invention uses deep learning for image recognition, achieving faster training time and higher prediction accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method, device, equipment and storage medium for image recognition based on deep learning. Background Art

[0002] In the field of industrial automation, image recognition plays an important role. The uses of image recognition include the detection of product defects and the determination of defect types, the identification and sorting of different products, and the automatic classification of products through machine vision algorithms, which can greatly improve the efficiency of industrial assembly lines, reduce labor costs, and improve quality. Nowadays, deep learning has reached and exceeded the accuracy of human eye recognition in certain specific tasks of image recognition, and has been widely used in the industrial field. Although deep learning has the advantage of high accuracy in image recognition, the learning direction is unclear, resulting in slow training time, and it is impossible to accurately judge abnormal samples beyond the labeled range, and there are even high-confidence erroneous predictions. Summary of the invention

[0003] The main purpose of the present invention is to provide an image recognition method, device, equipment and storage medium based on deep learning, aiming to solve the technical problems of slow training time and low prediction accuracy in the existing image recognition using deep learning.

[0004] In addition, to achieve the above object, the present invention also provides an image recognition method based on deep learning, and the image recognition method based on deep learning comprises the following steps:

[0005] Receiving a training image, obtaining a category label and a binary segmentation label of the training image, and performing feature extraction on the training image to obtain a feature space;

[0006] By using a preset convolutional neural network, attention prediction is performed on the feature space to obtain a target heat map, and a segmentation loss value is obtained according to the target heat map and the binary segmentation label;

[0007] Obtaining an embedding vector corresponding to the feature space and a category center corresponding to the category label, and calculating a distance loss value between the category center and the embedding vector by using a preset loss function;

[0008] The embedding vectors are integrated to obtain image categories, the category loss value between the image category and the category label is calculated, and the parameters of the preset convolutional neural network are adjusted according to the segmentation loss value, the distance loss value and the category loss value to complete model training.

[0009] Optionally, the step of extracting features from the training image to obtain a feature space includes:

[0010] The training image is input into a preset deep residual network, and features of the training image are extracted through the preset deep residual network to obtain a feature space of a preset size.

[0011] Optionally, the step of performing attention prediction on the feature space by using a preset convolutional neural network to obtain a target heat map includes:

[0012] By using a preset convolutional neural network, attention prediction is performed on the feature space to obtain attention values ​​of different areas of the feature space;

[0013] The attention value is limited to a preset interval by a preset nonlinear function to obtain a target heat map.

[0014] Optionally, the step of obtaining a segmentation loss value according to the target heat map and the binary segmentation label includes:

[0015] Performing a scaling operation on the binary image corresponding to the binary segmentation label so that the size of the binary image after the scaling operation is the same as the size of the target heat map;

[0016] The target heat map is compared with the binary image to obtain a segmentation loss value.

[0017] Optionally, the step of obtaining the embedding vector corresponding to the feature space includes:

[0018] Performing weighted processing on different regions of the feature space according to the attention values ​​in the target heat map;

[0019] Perform global average pooling on the weighted feature space to obtain an embedding vector corresponding to the feature space.

[0020] Optionally, the step of calculating the distance loss value between the category center and the embedding vector by using a preset loss function includes:

[0021] The embedded vector is brought closer to the category center corresponding to the category label by means of a preset loss function, and after the closer movement is completed, a distance loss value between the category center and the embedded vector after the closer movement is completed is calculated.

[0022] Optionally, after the step of adjusting parameters of the preset convolutional neural network to complete model training, the method further comprises:

[0023] Receiving a test image, and inputting the test image into the preset convolutional neural network to obtain a test vector;

[0024] Obtaining a feature center of the preset convolutional neural network, and comparing the test vector with the feature center;

[0025] According to the comparison result, a target feature center among the feature centers that is closest to the test vector is determined, and a target image category corresponding to the target feature center is used as an image category of the test image.

[0026] In addition, to achieve the above-mentioned purpose, the present invention further provides an image recognition device based on deep learning, and the image recognition device based on deep learning comprises:

[0027] A segmentation loss value calculation model is used to perform attention prediction on the feature space through a preset convolutional neural network to obtain a target heat map, and obtain a segmentation loss value according to the target heat map and the binary segmentation label;

[0028] A distance loss value calculation model is used to obtain the embedding vector corresponding to the feature space and the category center corresponding to the category label, and calculate the distance loss value between the category center and the embedding vector through a preset loss function;

[0029] The parameter adjustment model is used to integrate the embedding vector to obtain the image category, calculate the category loss value between the image category and the category label, and adjust the parameters of the preset convolutional neural network according to the segmentation loss value, the distance loss value and the category loss value to complete the model training.

[0030] In addition, to achieve the above-mentioned purpose, the present invention also provides an image recognition device based on deep learning, and the image recognition device based on deep learning includes: a memory, a processor, and a deep learning-based image recognition program stored in the memory and executable on the processor, and when the program is executed by the processor, the steps of the deep learning-based image recognition method as described above are implemented.

[0031] In addition, to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which a deep learning-based image recognition program is stored. When the deep learning-based image recognition program is executed by a processor, the steps of the deep learning-based image recognition method as described above are implemented.

[0032] The embodiment of the present invention proposes an image recognition method, device, equipment and storage medium based on deep learning. In the embodiment of the present invention, after receiving a training image, the category label and binary segmentation label of the pre-marked training image are obtained, and the training image is subjected to feature extraction to obtain a feature space, and then the feature space is predicted by a preset convolutional neural network to obtain a target heat map, and a segmentation loss value related to the prediction of the attention area is obtained according to the target heat map and the binary segmentation label, and then the embedding vector corresponding to the feature space and the category center corresponding to the category label are obtained, and the distance loss value between the category center and the embedding vector is calculated by a preset loss function, and then the embedding vector is integrated to obtain the image category, and the category loss value between the image category and the category label is calculated, and finally the preset convolutional neural network is parameter adjusted according to the segmentation loss value, the distance loss value and the category loss value to complete the model training. By introducing the attention mechanism, the present invention achieves faster training time and higher prediction accuracy when using deep learning for image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the hardware structure of an implementation of a deep learning-based image recognition device provided in an embodiment of the present invention;

[0034] Figure 2 It is a flowchart of the first embodiment of the image recognition method based on deep learning of the present invention;

[0035] Figure 3 It is a flowchart of a second embodiment of the image recognition method based on deep learning of the present invention;

[0036] Figure 4 This is a schematic diagram of the functional modules of an embodiment of an image recognition device based on deep learning of the present invention.

[0037] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0038] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0039] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0040] The deep learning-based image recognition terminal (also called terminal, device or terminal device) in the embodiment of the present invention can be a PC, or a mobile terminal device with a display function such as a smart phone, a tablet computer and a portable computer.

[0041] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0042] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the mobile terminal may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0043] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0044] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image recognition program based on deep learning.

[0045] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the deep learning-based image recognition program stored in the memory 1005. When the deep learning-based image recognition program is executed by the processor, the operations in the deep learning-based image recognition method provided in the following embodiments are implemented.

[0046] Based on the above-mentioned device hardware structure, an embodiment of the image recognition method based on deep learning of the present invention is proposed.

[0047] Reference Figure 2 In a first embodiment of the deep learning-based image recognition method of the present invention, the deep learning-based image recognition method includes:

[0048] Step S10, receiving a training image, obtaining a category label and a binary segmentation label of the training image, and performing feature extraction on the training image to obtain a feature space.

[0049] In this embodiment, the image recognition method based on deep learning is applied to image recognition. For example, in the field of industrial automation, various products produced by industrial production need to be quality inspected. Some quality inspections are based on product images. For example, the unevenness and cracks on the surface of the product. When inspecting the quality of these products, the images taken of the products can be observed. If the inspection item is whether the product has quality problems, then this is obviously a binary classification problem, that is, there are only two types of product images, product images with quality problems and product images without quality problems. If the inspection item is the type of product defects, then it is a multi-classification problem. There are many types of product images, including various types of product defect images and non-defective images. The training image in this embodiment refers to the image used for model training, which can be a defect image of the product or a defect-free product image. Generally, when performing model training, When performing model training, a large amount of training data is required. Therefore, there must be multiple training images here. It can be seen that before these training images are input into the deep learning system, these training images need to be labeled to indicate the type of training images. For example, the category label in this embodiment indicates the category of the training image. If the training image is a product defect image, the category label is the type of product defect. The number of categories is not limited, but for the convenience of classification, the number of categories here should not be too many. The binary segmentation label in this embodiment is used to identify the attention area, where the attention area refers to the degree of user attention in each area of ​​the training image, which can indirectly reflect the user's attention to a certain image area. The image area with high user attention is the area where the product often has defects. By introducing the attention mechanism, the image recognition accuracy can be improved and the image recognition speed can be accelerated.

[0050] After obtaining the category label and binary segmentation label of the training image, the deep residual network is used to extract features of the training image to obtain a feature space, and the width and height of the feature space are reduced to a certain proportion of the training image. For example, the width and height of the feature space are 1 / 16 of the width and height of the training image. It can be seen that the features of the training image are extracted, and the extracted features can be the color of the pixels in the training image, and the distribution of pixels of each color. The feature space in this embodiment is a three-dimensional feature space, which is composed of feature planes and feature constants. The distribution of features characterizes the characteristics of this type of image.

[0051] Step S20, performing attention prediction on the feature space through a preset convolutional neural network to obtain a target heat map, and obtaining a segmentation loss value based on the target heat map and the binary segmentation label.

[0052] After obtaining the feature space, the extracted deep features are predicted for the attention area. Specifically, by retrieving historical data, the heat map of the same scale of each area is determined, wherein the same scale refers to the same scale as the above-mentioned feature space after reduction. The heat map refers to the page area that visitors are keen on and the geographical area where the visitors are located in a special highlighted form. The user's behavior on the website is displayed using a heat spectrum map. The places with large views and large clicks are red, and the places with small views and small clicks are colorless and blue. This concept can be applied to the prediction of product defects through deep learning and the obtained product images, that is, the greater the user's attention, the more obvious the heat map corresponding to the area with more attention. It can be understood that the magnitude of the attention intensity can be reflected by the value. After obtaining the target heat map, it is also necessary to use the activation layer to limit the value of the target heat map within a certain range to filter out points or surfaces with abnormally large heat map values, and reduce the influence of these abnormal values ​​on the average heat map value corresponding to the attention area. It can be seen that when the value of the target heat map is restricted, there will be a certain deviation between the attention area corresponding to the target heat map and the attention area marked by the binary segmentation label. The calculated deviation is the segmentation loss value in this embodiment.

[0053] Step S30, obtaining the embedding vector corresponding to the feature space and the category center corresponding to the category label, and calculating the distance loss value between the category center and the embedding vector through a preset loss function.

[0054] It can be seen that by weighting the target heat map after passing through the activation layer, different attention areas with different deep features will be assigned different weights, that is, higher weights will be given to areas with higher attention scores. In this embodiment, the statistical method for the attention score can be to obtain the historical click rate of the user for each area of ​​the image or the number of times the mouse cursor stays, etc., which can reflect the historical information of the user's attention. It can be seen that the higher the attention score, the higher the user's attention to the area. After the weighting of the attention area is completed, the weighted deep features are globally averaged pooled. After the global average pooling is completed, the embedding vector can be obtained, wherein the global average pooling refers to All pixel values ​​of each channel map are averaged, and then a new one-by-one channel map is obtained. The embedding vector is pre-set to a specific dimension, such as one dimension. According to the above content, it is known that the category label has a certain number of categories, and each category corresponds to a category center, which is the absolute reference point of the corresponding category. That is, the closer the embedding vector corresponding to each image category is to the category center corresponding to the image category, the greater the probability that the image belongs to this image category. In this embodiment, the embedding vector corresponding to each image category is constrained by a loss function to be close to the category center corresponding to the image category, and finally the distance loss value between the embedding vector and the category center is calculated.

[0055] Step S40, integrating the embedding vector to obtain the image category, calculating the category loss value between the image category and the category label, and adjusting the parameters of the preset convolutional neural network according to the segmentation loss value, the distance loss value and the category loss value to complete the model training.

[0056] It can be seen that after obtaining the embedding vector, the embedding vector is integrated through the fully connected layer. After the integration is completed, the prediction result of the training image category can be obtained, wherein the prediction result may be an accurate result after screening. For example, there are five kinds of category labels mentioned above, namely four product defect images and one non-defective image. Then the prediction result of the category of the training image is five probability values, corresponding to five different category labels. The category label with the largest probability is the prediction result of the training image category. This prediction result may be different from the above-mentioned advance label calibration of the training image, and even if this prediction result is the same as the above-mentioned advance label calibration of the training image, there is still a certain difference between the two. This difference comes from the prediction result is a probability value. For example, if The above-mentioned labels of the training images are calibrated in advance as a type of defect image, and the prediction result of the training image is 95% as a type of defect image, then the difference between the two is 5%. If the prediction result is different from the label calibration, for example, if the labels of the training images are calibrated in advance as a type of defect image, the prediction result of the training image is 80% as a type of defect image, and 8% as a type of defect image, then the difference between the two is 92%. This difference is the category loss value in this embodiment. After obtaining the category loss value, the image recognition program based on deep learning will adjust the parameters of the preset convolutional neural network for training according to the segmentation loss value, the distance loss value and the category loss value. It can be seen that the adjusted parameters include the number of iterations, etc. After the parameter adjustment is completed, the model training process is completed.

[0057] Specifically, the steps refined in step S10 include:

[0058] Step a1: input the training image into a preset deep residual network, perform feature extraction on the training image through the preset deep residual network, and obtain a feature space of a preset size.

[0059] It can be seen that all training images have fixed dimensions and sizes. In this embodiment, after obtaining the training image, the training image is input into a preset deep residual network, and features of the training image are extracted through the preset deep residual network to obtain a feature space of a preset size.

[0060] Specifically, the steps of step S20 include:

[0061] Step b1, using a preset convolutional neural network to perform attention prediction on the feature space to obtain attention values ​​of different areas of the feature space.

[0062] Step b2, limiting the attention value to a preset interval by a preset nonlinear function to obtain a target heat map.

[0063] The attention prediction in this embodiment refers to calculating the feature space through a convolutional neural network to obtain a target heat map. Specifically, the attention values ​​of different areas of the feature space, that is, the importance, are calculated through a series of convolutional layers or fully connected layers in a preset convolutional neural network. After obtaining the attention values ​​of different areas of the feature space, the attention values ​​of different areas of the feature space can be limited to a preset interval through a preset nonlinear function to obtain the target heat map. This preset nonlinear function is also an activation function. For example, all input values ​​will be limited to between 0 and 1 after passing through the sigmoid function.

[0064] Specifically, the detailed steps of step S20 also include:

[0065] Step c1, performing a scaling operation on the binary image corresponding to the binary segmentation label so that the size of the binary image after the scaling operation is the same as the size of the target heat map.

[0066] Step c2: compare the target heat map with the binary map to obtain a segmentation loss value.

[0067] After the attention value is numerically limited, the image recognition program based on deep learning will obtain the size ratio of the plane size of the feature space and the image size of the training image. Specifically, after obtaining the category label and binary segmentation label of the training image, the training image is subjected to feature extraction through a deep residual network to obtain the feature space, and the width and height of the feature space are both reduced to a certain ratio of the training image. This ratio is the size ratio in this embodiment. After obtaining the size ratio, the segmented image corresponding to the binary segmentation label is scaled according to the size ratio. Finally, the filtered target heat map is compared with the binary image corresponding to the binary segmentation label obtained after scaling to obtain the segmentation loss value.

[0068] Specifically, the steps of step S30 include:

[0069] Step d1, performing weighted processing on different areas of the feature space according to the attention values ​​in the target heat map.

[0070] Step d2, performing global average pooling on the weighted feature space to obtain an embedding vector corresponding to the feature space.

[0071] Through the target heat map after the activation layer and numerical limitation, the spatial regions with different depth features are weighted, and the spatial regions with different depth features will be assigned different weights. The size of the weight indicates that the region has received higher attention. This higher attention is expressed by giving higher weights to regions with higher attention scores. After the depth features of the feature space are weighted, the weighted depth features are globally average pooled to obtain the embedding vector corresponding to the feature space. Specifically, the global average pooling process includes: assuming that the depth feature is a 4 by 4 pixel feature map, when performing global average pooling on it, the pool size is set to the feature map size, that is, 4 by 4, and the average value of all features in the depth feature map is taken to obtain the feature average value and the embedding vector corresponding to the feature average value. At this time, the obtained embedding vector is the dimensionality reduction vector.

[0072] Specifically, the detailed steps of step S30 also include:

[0073] Step e1, using a preset loss function, the embedding vector is brought closer to the category center corresponding to the category label, and after the approach is completed, the distance loss value between the category center and the embedded vector after the approach is completed is calculated.

[0074] It can be seen that by presetting the loss function, the above-obtained embedding vector can be constrained to move closer to the category center corresponding to the category label. The category center here corresponds to the category label, which is equivalent to the absolute reference of the category label. For example, if the training image is a type of defect image, then the category center is the absolute reference of the type of defect image. By continuously constraining the embedding vector to move closer to the category center, after the constraint is completed, the distance loss value between the category center and the embedded vector after the move is completed is calculated. It can be understood that the more the prediction result of the training image is similar to the category label, the smaller the distance loss value between the category center and the embedded vector after the move is completed.

[0075] In this embodiment, after receiving a training image, the category label and binary segmentation label of the pre-marked training image are obtained, and feature extraction is performed on the training image to obtain a feature space, and then attention prediction is performed on the feature space to obtain a target heat map, and based on the target heat map and the binary segmentation label, a segmentation loss value related to the attention area prediction is obtained, and then an embedding vector corresponding to the feature space and a category center corresponding to the category label are obtained, and a distance loss value between the category center and the embedding vector is calculated through a preset loss function, and then the embedding vector is integrated to obtain the image category, and the category loss value between the image category and the category label is calculated, and finally, according to the segmentation loss value, the distance loss value and the category loss value, the preset convolutional neural network is parameter adjusted to complete the model training. By introducing the attention mechanism, the present invention achieves faster training time and higher prediction accuracy when using deep learning for image recognition.

[0076] Further, refer to Figure 3 Based on the above-mentioned embodiments of the present invention, a second embodiment of the image recognition method based on deep learning of the present invention is proposed.

[0077] This embodiment is the step after step S40 in the first embodiment. The difference between this embodiment and the above-mentioned embodiment of the present invention is that:

[0078] Step S50, receiving a test image, and inputting the test image into the preset convolutional neural network to obtain a test vector.

[0079] Step S60, obtaining the feature center of the preset convolutional neural network, and comparing the test vector with the feature center.

[0080] Step S70: determining, according to the comparison result, a target feature center among the feature centers that is closest to the test vector, and taking the target image category corresponding to the target feature center as the image category of the test image.

[0081] After the model training is completed, the model testing phase is immediately followed. In the model testing phase, the preset convolutional neural network receives a test image for testing. The test image here is unlabeled with a category label and a binary segmentation label identifying the attention area. After receiving the test image, the preset convolutional neural network extracts features from the test image to obtain a feature vector, that is, the test vector in this embodiment. After obtaining the test vector, the image recognition program based on deep learning will further obtain the feature center maintained by the preset convolutional neural network during the training phase. The feature center here is related to the number of category labels marked during model training, that is, the number of feature centers is equal to the number of category labels marked during model training. After obtaining the test vector and the feature center, the image recognition program based on deep learning will compare the test vector with all the feature centers one by one, and determine the target feature center closest to the test vector among all the feature centers based on the comparison result, and use the target image category corresponding to the target feature center as the image category of the test image, thereby completing the image classification of the test image. If the distance between the target feature center and the test vector is greater than a certain threshold, it indicates that the test image is an abnormal image, for example, other images not related to the test.

[0082] In this embodiment, by testing the trained preset convolutional neural network, while verifying the training results of the preset convolutional neural network, the preset convolutional neural network can also be improved to achieve higher accuracy image prediction.

[0083] In addition, refer to Figure 4The embodiment of the present invention further provides an image recognition device based on deep learning, and the image recognition device based on deep learning includes:

[0084] The feature extraction model 10 is used to receive a training image, obtain a category label and a binary segmentation label of the training image, and perform feature extraction on the training image to obtain a feature space;

[0085] A segmentation loss value calculation model 20 is used to perform attention prediction on the feature space through a preset convolutional neural network to obtain a target heat map, and obtain a segmentation loss value according to the target heat map and the binary segmentation label;

[0086] A distance loss value calculation model 30 is used to obtain the embedding vector corresponding to the feature space and the category center corresponding to the category label, and calculate the distance loss value between the category center and the embedding vector through a preset loss function;

[0087] The parameter adjustment model 40 is used to integrate the embedding vector to obtain the image category, calculate the category loss value between the image category and the category label, and adjust the parameters of the preset convolutional neural network according to the segmentation loss value, the distance loss value and the category loss value to complete model training.

[0088] Optionally, the feature extraction model 10 includes:

[0089] The feature extraction unit is used to input the training image into a preset deep residual network, perform feature extraction on the training image through the preset deep residual network, and obtain a feature space of a preset size.

[0090] Optionally, the segmentation loss value calculation model 20 includes:

[0091] An attention prediction unit, used to perform attention prediction on the feature space through a preset convolutional neural network to obtain attention values ​​of different areas of the feature space;

[0092] The value limiting unit is used to limit the attention value to a preset interval through a preset nonlinear function to obtain a target heat map.

[0093] Optionally, the segmentation loss value calculation model 20 further includes:

[0094] A scaling unit, used for performing a scaling operation on the binary image corresponding to the binary segmentation label so that the size of the binary image after the scaling operation is the same as the size of the target heat map;

[0095] A comparison unit is used to compare the target heat map with the binary image to obtain a segmentation loss value.

[0096] Optionally, the distance loss value calculation model 30 includes:

[0097] A weighted processing unit, used for performing weighted processing on different areas of the feature space according to the attention value in the target heat map;

[0098] The average pooling unit is used to perform global average pooling on the weighted feature space to obtain an embedding vector corresponding to the feature space.

[0099] Optionally, the distance loss value calculation model 30 further includes:

[0100] The approaching unit is used to approach the embedding vector to the category center corresponding to the category label through a preset loss function, and after the approaching is completed, calculate the distance loss value between the category center and the embedded vector after the approaching is completed.

[0101] Optionally, the image recognition device based on deep learning further includes:

[0102] A test image receiving module, used for receiving a test image and inputting the test image into the preset convolutional neural network to obtain a test vector;

[0103] A feature center acquisition module, used to acquire the feature center of the preset convolutional neural network and compare the test vector with the feature center;

[0104] A determination module is used to determine the target feature center that is closest to the test vector among the feature centers according to the comparison result, and use the target image category corresponding to the target feature center as the image category of the test image.

[0105] In addition, an embodiment of the present invention further proposes a storage medium, on which a deep learning-based image recognition program is stored. When the deep learning-based image recognition program is executed by a processor, the operations in the deep learning-based image recognition method provided in the above embodiment are implemented.

[0106] The methods executed by the above-mentioned program modules can refer to the various embodiments of the method of the present invention, and will not be described in detail here.

[0107] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity / operation / object from another entity / operation / object, and do not necessarily require or imply any such actual relationship or order between these entities / operations / objects; the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0108] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. The device embodiment described above is only illustrative, and the units described as separate components may or may not be physically separated. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the present invention. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0109] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the image recognition method based on deep learning described in each embodiment of the present invention.

[0111] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image recognition method based on deep learning, It is characterized in that The image recognition method based on deep learning comprises the following steps: Receiving a training image, obtaining a category label and a binary segmentation label of the training image, and performing feature extraction on the training image to obtain a feature space; By using a preset convolutional neural network, the feature space is predicted for attention to obtain a target heat map, and a segmentation loss value is obtained according to the target heat map and the binary segmentation label, wherein the feature space is predicted for attention by using a preset convolutional neural network to obtain attention values ​​of different attention areas of the feature space; the attention values ​​are limited to a preset interval by using a preset nonlinear function to filter out points or surfaces with abnormally large attention, and heat maps of the same scale of each attention area are determined to obtain a target heat map, where the same scale refers to the same scale as the feature space after being reduced, and the heat map refers to the page area that visitors are keen on and the geographical area where the visitors are located in a special highlighted form, and the attention area refers to the degree of attention received by users in each area of ​​the training image, and the heat map corresponding to the area with a larger attention value is more obvious; Obtaining an embedding vector corresponding to the feature space and a category center corresponding to the category label, and calculating a distance loss value between the category center and the embedding vector through a preset loss function, wherein different regions of the feature space are weighted according to the attention value in the target heat map, and global average pooling is performed on the weighted feature space to obtain an embedding vector corresponding to the feature space; The embedding vectors are integrated to obtain image categories, the category loss value between the image category and the category label is calculated, and the parameters of the preset convolutional neural network are adjusted according to the segmentation loss value, the distance loss value and the category loss value to complete model training.

2. The image recognition method based on deep learning as claimed in claim 1, It is characterized in that The step of extracting features from the training image to obtain a feature space comprises: The training image is input into a preset deep residual network, and features of the training image are extracted through the preset deep residual network to obtain a feature space of a preset size.

3. The image recognition method based on deep learning as claimed in claim 1, It is characterized in that The step of obtaining the segmentation loss value according to the target heat map and the binary segmentation label comprises: Performing a scaling operation on the binary image corresponding to the binary segmentation label so that the size of the binary image after the scaling operation is the same as the size of the target heat map; The target heat map is compared with the binary image to obtain a segmentation loss value.

4. The image recognition method based on deep learning as claimed in claim 1, It is characterized in that The step of calculating the distance loss value between the category center and the embedding vector by using a preset loss function comprises: The embedded vector is brought closer to the category center corresponding to the category label by means of a preset loss function, and after the closer movement is completed, a distance loss value between the category center and the embedded vector after the closer movement is completed is calculated.

5. The image recognition method based on deep learning as claimed in claim 1, It is characterized in that After the step of adjusting the parameters of the preset convolutional neural network to complete the model training, the method further comprises: Receiving a test image, and inputting the test image into the preset convolutional neural network to obtain a test vector; Obtaining a feature center of the preset convolutional neural network, and comparing the test vector with the feature center; According to the comparison result, a target feature center among the feature centers that is closest to the test vector is determined, and a target image category corresponding to the target feature center is used as an image category of the test image.

6. An image recognition device based on deep learning, It is characterized in that The image recognition device based on deep learning includes: A feature extraction module is used to receive a training image, obtain a category label and a binary segmentation label of the training image, and perform feature extraction on the training image to obtain a feature space; The segmentation loss value calculation module is used to perform attention prediction on the feature space through a preset convolutional neural network to obtain a target heat map, and obtain a segmentation loss value according to the target heat map and the binary segmentation label, wherein the feature space is predicted for attention through a preset convolutional neural network to obtain attention values ​​of different attention areas of the feature space; the attention values ​​are limited to a preset interval through a preset nonlinear function to filter out points or surfaces with abnormally large attention, and the heat map of the same scale of each attention area is determined to obtain a target heat map. The same scale means the same scale as the feature space after reduction. The heat map refers to the page area that visitors are keen on and the geographical area where the visitors are located in a special highlighted form. The attention area refers to the degree of attention received by users in each area of ​​the training image. The heat map corresponding to the area with a larger attention value is more obvious. A distance loss value calculation module is used to obtain the embedding vector corresponding to the feature space and the category center corresponding to the category label, and calculate the distance loss value between the category center and the embedding vector through a preset loss function, wherein different areas of the feature space are weighted according to the attention value in the target heat map, and the weighted feature space is globally averaged pooled to obtain the embedding vector corresponding to the feature space; A parameter adjustment module is used to integrate the embedding vector to obtain the image category, calculate the category loss value between the image category and the category label, and adjust the parameters of the preset convolutional neural network according to the segmentation loss value, the distance loss value and the category loss value to complete model training.

7. An image recognition device based on deep learning, It is characterized in that The deep learning-based image recognition device includes: a memory, a processor, and a deep learning-based image recognition program stored in the memory and executable on the processor. When the deep learning-based image recognition program is executed by the processor, the steps of the deep learning-based image recognition method as described in any one of claims 1 to 5 are implemented.

8. A storage medium, It is characterized in that The storage medium stores a deep learning-based image recognition program, which, when executed by a processor, implements the steps of the deep learning-based image recognition method according to any one of claims 1 to 5.

Citation Information

Patent Citations

  • General target detection method of adaptive attention guidance mechanism

    CN111259930A

  • Small target detection method for context feature fusion screening based on attention mechanism

    CN111275688A