Image recognition method and device based on hybrid neural network

Abstract

The invention relates to the technical field of image recognition, in particular to an image recognition method and device based on a hybrid neural network, and the method comprises the steps: obtaining an image data set; constructing a hybrid neural network model, and training the hybrid neural network model by using the image data set; wherein the hybrid neural network module comprises an equal mapping sub-module, a feature extraction sub-module and a feature re-calibration sub-module; the feature extraction sub-module is used for extracting features of multiple dimensions from the input information and fusing the features of the multiple dimensions to obtain a fused feature map; the feature re-calibration sub-module is used for optimizing the weight of the fused feature map and adaptively adjusting the weight of the fused feature map through learning to obtain an optimized feature map; the equality mapping sub-module is used for reserving input information so as to obtain an output feature map according to the input information and the optimized feature map; and identifying a test image based on the trained hybrid neural network model.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an image recognition method and device based on a hybrid neural network. Background Art

[0002] Compared with traditional optical remote sensing equipment, high-resolution imaging systems have the characteristics of active imaging, wide band, and no weather influence, and can conduct stable and continuous earth observations around the clock, so they are widely used in various fields. However, some high-resolution imaging systems generate images with serious coherent speckle noise and even geometric distortion, resulting in poor visual readability of the images, making it very difficult to identify targets in the images with the naked eye.

[0003] In recent years, a large number of traditional machine learning algorithms have been introduced into the field of image recognition and have made some progress. However, such methods require artificial design of classifiers and manual feature extraction, and the processing process is very cumbersome. On the other hand, low-order visual features extracted based on basic information such as image edges, textures, gradients, and spectra are also difficult to fully characterize the original image information, resulting in limited recognition effects.

[0004] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the existing image recognition method requires artificial design of classifiers and manual feature extraction, and the processing process is very cumbersome.

[0006] The present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides an image recognition method based on a hybrid neural network, comprising:

[0008] Get image dataset;

[0009] Constructing a hybrid neural network model, and using the image data set to train the hybrid neural network model; wherein the hybrid neural network model includes a plurality of hybrid neural network modules, the output feature map of the previous hybrid neural network module is used as the input information of the next hybrid neural network module, and the hybrid neural network module includes: an identity mapping submodule, a feature extraction submodule and a feature recalibration submodule; the feature extraction submodule is used to extract features of multiple dimensions from the input information, and fuse the features of the multiple dimensions to obtain a fused feature map; the feature recalibration submodule is used to optimize the weight of the fused feature map, and adaptively adjust the weight of the fused feature map through learning to obtain an optimized feature map; the identity mapping submodule is used to retain the input information, so as to obtain an output feature map according to the input information and the optimized feature map;

[0010] The test image is recognized based on the trained hybrid neural network model.

[0011] Preferably, after acquiring the image data set, the step further includes:

[0012] Preprocessing the image data set, wherein the preprocessing includes image denoising, data enhancement, unifying the size and normalizing the images in the image data set;

[0013] The preprocessed image data set is randomly divided into a training set, a validation set and a test set; wherein the training set is used for training the hybrid neural network model, the validation set is used to judge the convergence of the hybrid neural network model and hyperparameter tuning, and the test set is used to evaluate the generalization performance of the hybrid neural network model.

[0014] Preferably, the first formula is used to perform image denoising, wherein the first formula is:

[0015]

[0016] in, is the image value after removing the coherent speckle noise, I(t) is the image to be denoised, is the mean of the neighborhood window of I(t) pixels, and w(t) is the weight coefficient;

[0017] Among them, C U represents the coherent speckle noise, C I (t) represents the standard deviation coefficient of image I(t).

[0018] Preferably, the feature extraction submodule includes a plurality of feature extraction channels and convolution kernel splicing layers; each feature extraction channel includes at least one convolution kernel, and one feature extraction channel also includes a maximum pooling layer;

[0019] The multiple feature extraction channels are used to extract features of multiple dimensions;

[0020] The convolution kernel splicing layer is used to fuse the features of the multiple dimensions to obtain a fused feature map.

[0021] Preferably, the feature recalibration submodule comprises: a global pooling layer, two fully connected layers, an activation function layer and a Scale layer connected in sequence; the global pooling layer is connected to the convolution kernel splicing layer, and the Scale layer is also connected to the convolution kernel splicing layer;

[0022] The global pooling layer is used to compress the fused feature map into a feature vector.

[0023] The two fully connected layers and the activation function layer are used to learn the weight of each channel according to the feature vector to capture the relationship between channels and obtain a weight vector for each channel;

[0024] The Scale layer is used to apply the weight vector of each channel to each channel in the fused feature map to obtain an optimized feature map.

[0025] Preferably, the hybrid neural network model further includes an input layer, a convolutional layer, a maximum pooling layer, a global pooling layer, a fully connected layer and a softmax layer;

[0026] The input layer, the convolution layer and the maximum pooling layer are connected in sequence, the maximum pooling layer is connected to the first hybrid neural network module, the last hybrid neural network module is connected to the global pooling layer, and the global pooling layer is connected to the fully connected layer and the softmax layer in sequence;

[0027] The identifying of the test image based on the trained hybrid neural network model includes:

[0028] The convolution layer performs feature extraction on the test image to obtain an initial feature map; the maximum pooling layer performs dimensionality reduction processing on the initial feature map to obtain a compressed feature map; wherein the compressed feature map serves as input information of the first hybrid neural network module;

[0029] Each hybrid neural network module processes the received input information until the last hybrid neural network module outputs a corresponding output feature map, wherein the output feature map of the previous hybrid neural network module serves as input information of the next hybrid neural network module;

[0030] The global pooling layer performs an average operation on each channel of the received output feature map, calculates the average value of all values ​​on each channel, and obtains a one-dimensional vector;

[0031] The fully connected layer and the softmax layer output recognition results according to the one-dimensional vector.

[0032] Preferably, before training the hybrid neural network model, the method further comprises:

[0033] Xavier is used to initialize the hybrid neural network model.

[0034] Preferably, the adopting the image data set to train the hybrid neural network model specifically includes:

[0035] The initialized hybrid neural network model is trained on the training set using the Adam optimizer;

[0036] The optimized hybrid neural network model is validated on the validation set and hyperparameters are tuned.

[0037] In a second aspect, the present invention provides a device for image recognition based on a hybrid neural network, which is used to implement the image recognition method based on a hybrid neural network described in the first aspect, and the device comprises:

[0038] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the image recognition method based on the hybrid neural network described in the first aspect.

[0039] In a third aspect, the present invention provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the image recognition method based on the hybrid neural network as described in the first aspect.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides an image recognition method based on a hybrid neural network, designs a new hybrid neural network model, introduces an identity mapping submodule, a feature extraction submodule and a feature recalibration submodule into the hybrid neural network model, utilizes the powerful representation ability of deep learning including the above submodules, automatically extracts features to realize end-to-end image recognition, eliminates the steps of manually designing classifiers and feature engineering, and has improvements in recognition speed and recognition accuracy compared with traditional machine learning algorithms, effectively improving the performance and efficiency of the hybrid neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is a schematic diagram of the overall process of an image recognition method based on a hybrid neural network provided by an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of step 1 of an image recognition method based on a hybrid neural network provided by an embodiment of the present invention;

[0044] Figure 3 It is a schematic diagram of an IISE network model structure of an image recognition method based on a hybrid neural network provided by an embodiment of the present invention;

[0045] Figure 4 is a schematic diagram of step 2 of an image recognition method based on a hybrid neural network provided by an embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of the method program logic of an image recognition method based on a hybrid neural network provided by an embodiment of the present invention;

[0047] Figure 6 It is a structural schematic diagram of an image recognition device based on a hybrid neural network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "examples", "specific examples" or "some examples" and the like are intended to indicate that specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0050] In the description of the present invention, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more. In addition, for example, the same type of nouns may be described as two independent individuals by adding "A" and "B" at the end. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the same type of individuals for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.

[0051] When describing some embodiments, the expressions "coupling", "coupling" and "connection" and their derivatives may be used. For example, when describing some embodiments, the term "connection" may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term "coupling" may be used to indicate that two or more components are in direct physical or electrical contact. However, the terms "connection" or "coupling" may also refer to two or more components that are not in direct contact with each other, but still cooperate or interact with each other, such as "optical path coupling", "wireless connection", etc. The embodiments disclosed herein are not necessarily limited to the contents of the present invention.

[0052] In the description of the present invention, the expression "A and / or B" (where A and B are used to formally represent specific characteristic contents) will be involved, and the corresponding expressions include the following three combinations: only A, only B, and a combination of A and B.

[0053] As used herein, "about," "substantially," or "approximately" includes the stated value and an average value that is within an acceptable range of deviation from the particular value as determined by one of ordinary skill in the art taking into account the measurements in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0054] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0055] Embodiment 1:

[0056] Embodiment 1 of the present invention provides an image recognition method based on a hybrid neural network, such as Figure 1 As shown, including:

[0057] In step S1, an image dataset is acquired.

[0058] In one embodiment, the image data set includes a plurality of images, which may be synthetic aperture radar (SAR) images.

[0059] In step S2, a hybrid neural network model is constructed, and the hybrid neural network model is trained using the image data set.

[0060] Among them, the hybrid neural network model includes multiple hybrid neural network modules, the output feature map of the previous hybrid neural network module is used as the input information of the next hybrid neural network module, and the hybrid neural network module includes: an identity mapping submodule, a feature extraction submodule and a feature recalibration submodule; the feature extraction submodule is used to extract features of multiple dimensions from the input information, and fuse the features of the multiple dimensions to obtain a fused feature map; the feature recalibration submodule is used to optimize the weight of the fused feature map, and adaptively adjust the weight of the fused feature map through learning to obtain an optimized feature map; the identity mapping submodule is used to retain the input information so as to obtain an output feature map based on the input information and the optimized feature map.

[0061] In this embodiment, the number of hybrid neural network modules included in the hybrid neural network model is not specifically limited, and may be 3 or 4. Figure 3 For example, the hybrid neural network model has three hybrid neural network modules.

[0062] In step S3, the test image is recognized based on the trained hybrid neural network model.

[0063] The present invention provides an image recognition method based on a hybrid neural network, designs a new hybrid neural network model, introduces an identity mapping submodule, a feature extraction submodule and a feature recalibration submodule into the hybrid neural network model, utilizes the powerful representation ability of deep learning of the hybrid neural network model including the above submodules, automatically extracts features to realize end-to-end image recognition, eliminates the links of manually designing classifiers and feature engineering, and has improvements in recognition speed and recognition accuracy compared with traditional machine learning algorithms, effectively improving the performance and efficiency of the hybrid neural network model.

[0064] In order to fully illustrate the solution provided by the present invention, each step of the above method is described in detail below.

[0065] In the above step S1, the image data set is obtained, such as Figure 2 As shown, specifically including:

[0066] In step S10, the image dataset is preprocessed, and the preprocessing includes image denoising, data enhancement, unification of size and normalization of the images in the image dataset.

[0067] In step S11, the preprocessed image data set is randomly divided into a training set, a validation set and a test set; wherein the training set is used for training the hybrid neural network model, the validation set is used to judge the convergence of the hybrid neural network model and hyperparameter tuning, and the test set is used to evaluate the generalization performance of the hybrid neural network model.

[0068] For example, there will be coherent speckle noise in the image. Coherent speckle noise is a multiplicative noise. Multiplicative noise is an important type of noise in communication and signal processing. Its relationship with the signal is nonlinear and depends on the existence of the signal. The impact of multiplicative noise on the communication system is complex and difficult to handle, and special technologies are required to suppress and eliminate it. In the fields of wireless communication and image processing, the existence of multiplicative noise has an important impact on the transmission and reception quality of the signal. Its mathematical model can be expressed by the following formula: I(t) = R(t) × U(t), where I(t) represents the received image and R(t) represents the clear object image that should actually be observed. U(t) represents the coherent speckle noise formed in the imaging process, which obeys the gamma distribution with a mean of 1 and a variance of 1 / ENL, and ENL is the equivalent number of views.

[0069] Specifically, the image denoising adopts a first formula for image denoising, wherein the first formula is: In the formula represents the image value after removing the coherent speckle noise, I(t) is the image to be denoised, It represents the mean of the neighborhood window of I(t) pixels, and w(t) is the weight coefficient.

[0070] The weight coefficient w(t) can be calculated by the following formula: Where C U represents the coherent speckle noise U(t), C I (t) represents the standard deviation coefficient of image I(t).

[0071] The standard deviation coefficient C U and C I (t) can be calculated by the following formula: Where σ u and are the standard deviation and mean of the coherent speckle noise U(t), σ I (t) is the standard deviation of the pixel neighborhood window of image I(t).

[0072] The data enhancement includes flipping, rotating, scaling and translating the image, and the uniform size is to crop all images to the same size. In the process of data acquisition, large-scale image data is difficult to obtain. If there is not enough data to match the complex network model, it is easy to cause overfitting of the network model. In order to solve this problem, in an embodiment of the present invention, the existing data set is expanded by data enhancement. The data enhancement method used includes flipping, rotating, scaling, translating, etc. the original image. The expanded data set has a total of 10,000 images.

[0073] In the acquired image dataset, the image sizes of each type of target may not be exactly the same, and there may be multiple sizes such as 100×100, 128×128, 158×158 or 177×178. Before training the hybrid neural network model, it is necessary to crop the input image. In most image datasets, by observing the image, it can be found that the effective information of each type of sample is concentrated in the center of the image, so all images can be cropped into a uniform size, such as 100×100.

[0074] After obtaining an image data set of uniform size, the image may be affected by environmental factors such as light and shadow, resulting in large differences in image data in different areas. Or in image recognition, the image may come from different resolutions and viewing angles, resulting in some feature values ​​in the image being too large or too small, which is not conducive to the training and learning of the neural network. Therefore, in an embodiment of the present invention, the original image data is normalized to make the data distribution more uniform, which is conducive to the training of the model, as shown in the following formula: Where x is the original image, μ and σ are the mean and standard deviation of the image, and x * is the normalized image.

[0075] Normalizing image data can accelerate neural network training. For example, scaling all input data to a uniform range (such as [0, 1] or [-1, 1]) can reduce the impact of inputs of different dimensions on the training process, making the network easier to converge, improving the convergence of the neural network, and preventing gradient vanishing or exploding problems caused by input data that is too large or too small, thereby ensuring that the network can stably learn effective feature representations and eliminate the impact of uneven light in the image. Normalization can eliminate the differences caused by uneven light, making image data more consistent and convenient for subsequent processing and analysis. At the same time, the normalized data has a similar scale, which can make the model more fair when processing different images, and will not dominate the learning process due to some feature values ​​that are too large or too small, thereby improving the generalization ability and accuracy of the model, reducing the demand for computing resources, making images closer at the data level, facilitating subsequent feature extraction and classification recognition, and reducing the impact of numerical differences between different features on algorithm performance.

[0076] In the above step S11, the preprocessed image data set is divided, specifically including randomly dividing the preprocessed image data set into a training set, a validation set and a test set; wherein the training set is used for training the hybrid neural network model, the validation set is used to judge the convergence of the hybrid neural network model and the hyperparameter tuning, and the test set is used to evaluate the generalization performance of the hybrid neural network model. In one embodiment, the training set, the validation set and the test set data are randomly extracted from the original image data, and the ratio can be 60%:20%:20% respectively. For example, if there are 10,000 images in total, the training set data contains 6,000 images, the validation set data contains 2,000 images, and the test set data contains 2,000 images.

[0077] like Figure 3 As shown, the feature extraction submodule includes multiple feature extraction channels and convolution kernel splicing layers; each feature extraction channel includes at least one convolution kernel, and one feature extraction channel also includes a maximum pooling layer; the multiple feature extraction channels are used to extract features of multiple dimensions; the convolution kernel splicing layer is used to fuse the features of the multiple dimensions to obtain a fused feature map. Figure 3 For example, the feature extraction submodule includes 4 feature extraction channels. From left to right, the first feature extraction channel includes a 1×1 convolution kernel, the second feature extraction channel includes a 1×1 convolution kernel and a 3×3 convolution kernel, the third feature extraction channel includes a 1×1 convolution kernel and a 5×5 convolution kernel, and the fourth feature extraction channel includes a 3×3 maximum pooling layer and a 1×1 convolution kernel.

[0078] The feature recalibration submodule includes: a global pooling layer, two fully connected layers, an activation function layer and a Scale layer connected in sequence; the global pooling layer is connected to the convolution kernel splicing layer, and the Scale layer is also connected to the convolution kernel splicing layer; the global pooling layer is used to compress the fused feature map into a feature vector, the two fully connected layers and the activation function layer are used to learn the weight of each channel according to the feature vector to capture the relationship between channels and obtain the weight vector of each channel; the Scale layer is used to apply the weight vector of each channel to each channel in the fused feature map to obtain an optimized feature map.

[0079] Continue reading Figure 3 The hybrid neural network model also includes an input layer, a convolutional layer, a maximum pooling layer, a global pooling layer, a fully connected layer and a softmax layer.

[0080] The input layer, the convolution layer and the maximum pooling layer are connected in sequence, the maximum pooling layer is connected to the first hybrid neural network module, the last hybrid neural network module is connected to the global pooling layer, and the global pooling layer is connected to the fully connected layer and the softmax layer in sequence.

[0081] In one embodiment, step S3 includes: the convolution layer performs feature extraction on the test image to obtain an initial feature map; the maximum pooling layer performs dimensionality reduction processing on the initial feature map to obtain a compressed feature map; wherein the compressed feature map serves as input information of the first hybrid neural network module.

[0082] Each hybrid neural network module processes the received input information until the last hybrid neural network module outputs a corresponding output feature map, wherein the output feature map of the previous hybrid neural network module serves as input information for the next hybrid neural network module.

[0083] The global pooling layer performs an average operation on each channel of the received output feature map, calculates the average value of all values ​​on each channel, and obtains a one-dimensional vector; the fully connected layer and the softmax layer output recognition results according to the one-dimensional vector.

[0084] In the above step S2, before training the hybrid neural network model, it also includes: initializing the hybrid neural network model using Xavier, wherein the initial learning rate can be set to 0.001, which decays with the number of iterations; the batch size can be set to 128; and the number of training rounds can be set to 200.

[0085] In the above step S2, the hybrid neural network model is trained, validated and hyperparameterized on the image dataset, such as Figure 4 As shown, specifically including:

[0086] In step S20, the initialized hybrid neural network model is trained on the training set using an Adam optimizer.

[0087] Specifically, after the hybrid neural network model is initialized, the Adam optimizer is used for optimization, and the loss function used in the optimization process is the cross entropy loss function.

[0088] In step S21, the optimized hybrid neural network model is validated on a validation set and hyperparameters are tuned.

[0089] In the above step S3, the test image is recognized based on the trained hybrid neural network model, including: the hybrid neural network model is tested on the test set of the image data set, and the predicted result is a set of vectors, which represent the probability that the test image belongs to each category, and the category with the largest probability value is the recognition result of the image.

[0090] In the above scheme, the identity mapping submodule plays a key role, including: alleviating the vanishing or exploding gradient, adding the input directly to the output of the residual block through the jump connection, so that the gradient can be directly propagated to the previous layer, thereby alleviating the vanishing or exploding gradient problem in the deep network training; simplifying the network training, even if the transformation in the residual block (such as the convolution layer) is trained to be close to zero, the output of the residual block is still close to the input. This feature makes it easier for the network to retain the performance of the shallow network during training, and also simplifies the learning process of the network. Based on this, the identity mapping submodule can retain the input information of the previous layer, so that the target of network fitting becomes the residual. Intuitively, the residual needs to learn less content, is generally smaller, and is easier to learn. When the residual is zero, it is equivalent to doing an identity mapping, ensuring that the network performance will at least not decrease. Therefore, the introduction of the identity mapping submodule can effectively improve the network learning effect.

[0091] The feature extraction submodule consists of multiple convolutional layers and pooling layers, which can process input data in parallel and merge their outputs to generate the final output; and by introducing a 1x1 convolutional layer as a bottleneck layer, the number of input channels of the subsequent convolutional layers is reduced, thereby significantly reducing the amount of calculation and the number of parameters; for different sizes of convolutional kernels (such as 3x3, 5x5) and pooling layers (such as 3x3 maximum pooling) work in parallel to extract features of different scales, enhancing the multi-scale capability of the model. Based on this, the feature extraction submodule is the core basic module in the network, used for feature extraction, for example, using convolutional kernels of sizes 1, 3, and 5 to extract features of receptive fields of different sizes, padding is set to 0, 1, and 2, feature maps of the same size can be obtained, and these feature maps are spliced ​​together along the channel dimension to complete feature fusion. The feature extraction submodule uses 1×1 convolutional kernels multiple times, and its main function is to reduce the dimension and reduce the number of parameters.

[0092] The feature recalibration submodule improves the model's sensitivity to important features by recalibrating different channels of the feature map. Its specific operations include: Squeeze operation, which compresses the feature map of each channel into a scalar through global average pooling to obtain global information at the channel level; Excitation operation: through two fully connected layers (usually containing ReLU activation function) and a Sigmoid activation function, the channel-level global information obtained by the Squeeze operation is nonlinearly transformed to generate the weight of each channel; Scale operation, which multiplies the weight obtained by the Excitation operation with each channel of the original feature map to achieve feature recalibration. Based on the above operations, the model can adaptively adjust the feature importance of different channels, thereby improving the model's sensitivity to important features and thus improving the performance of the model.

[0093] After the last hybrid neural network module, a global pooling layer is connected. Here, the fully connected layer with huge computational complexity is no longer used, which can greatly reduce the number of parameters. After the global pooling layer, the fully connected layer and the softmax layer are finally output. Therefore, in the solution provided by the embodiment of the present invention, the identity mapping submodule, the feature extraction submodule and the feature recalibration submodule are introduced to effectively improve the performance and efficiency of the hybrid neural network model.

[0094] Combining the above methods, such as Figure 5 As shown, this is the overall process logic diagram of the image recognition method based on hybrid neural network.

[0095] Embodiment 2:

[0096] Based on the above method provided by the embodiment of the present invention, the embodiment of the present invention also provides a device for image recognition based on a hybrid neural network, which is used in the image recognition method based on a hybrid neural network described in the above scheme, such as Figure 6 As shown, the device includes: at least one processor 10; and a memory 11 communicatively connected to the at least one processor 10; wherein the memory 11 stores instructions executable by the at least one processor 10, and the instructions are executed by the processor 10 to execute the image recognition method based on the hybrid neural network described in the above method.

[0097] like Figure 6 , which is a schematic diagram of the architecture of a device according to an embodiment of the present invention. The device according to this embodiment includes one or more processors 10 and a memory 11. Figure 6 A processor 10 is taken as an example.

[0098] The processor 10 and the memory 11 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0099] The memory 11 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer executable programs, such as the method in Embodiment 1. The processor 10 executes the method by running the non-volatile software programs and instructions stored in the memory 11 .

[0100] The memory 11 may include a high-speed random access memory 11, and may also include a non-volatile memory 11, such as at least one disk memory 11, a flash memory device, or other non-volatile solid-state memory 11. In some embodiments, the memory 11 may optionally include a memory 11 remotely disposed relative to the processor 10, and these remote memories 11 may be connected to the processor 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] The program instructions / modules are stored in the memory 11, and when executed by the one or more processors 10, the method in the above embodiment 1 is executed, for example, the method described above is executed. Figure 1-Figure 5 The steps shown.

[0102] In addition to the above-mentioned method and apparatus, an embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the image recognition method based on the hybrid neural network as described in the above-mentioned scheme.

[0103] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0104] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An image recognition method based on a hybrid neural network, characterized in that: include: Get image dataset; Constructing a hybrid neural network model, and using the image data set to train the hybrid neural network model; wherein the hybrid neural network model includes a plurality of hybrid neural network modules, the output feature map of the previous hybrid neural network module is used as the input information of the next hybrid neural network module, and the hybrid neural network module includes: an identity mapping submodule, a feature extraction submodule and a feature recalibration submodule; the feature extraction submodule is used to extract features of multiple dimensions from the input information, and fuse the features of the multiple dimensions to obtain a fused feature map; the feature recalibration submodule is used to optimize the weight of the fused feature map, and adaptively adjust the weight of the fused feature map through learning to obtain an optimized feature map; the identity mapping submodule is used to retain the input information, so as to obtain an output feature map according to the input information and the optimized feature map; The test image is recognized based on the trained hybrid neural network model.

2. The image recognition method based on hybrid neural network according to claim 1, characterized in that: The step of obtaining the image data set further includes: Preprocessing the image data set, wherein the preprocessing includes image denoising, data enhancement, unifying the size and normalizing the images in the image data set; The preprocessed image data set is randomly divided into a training set, a validation set and a test set; wherein the training set is used for training the hybrid neural network model, the validation set is used to judge the convergence of the hybrid neural network model and hyperparameter tuning, and the test set is used to evaluate the generalization performance of the hybrid neural network model.

3. The image recognition method based on hybrid neural network according to claim 2, characterized in that: The first formula is used to perform image denoising; The first formula is: in, is the image value after removing the coherent speckle noise, I(t) is the image to be denoised, is the mean of the neighborhood window of I(t) pixels, and w(t) is the weight coefficient; Among them, C U represents the coherent speckle noise, C I (t) represents the standard deviation coefficient of image I(t).

4. The image recognition method based on hybrid neural network according to claim 1, characterized in that: The feature extraction submodule includes a plurality of feature extraction channels and convolution kernel splicing layers; each feature extraction channel includes at least one convolution kernel, and one feature extraction channel also includes a maximum pooling layer; The multiple feature extraction channels are used to extract features of multiple dimensions; The convolution kernel splicing layer is used to fuse the features of the multiple dimensions to obtain a fused feature map.

5. The image recognition method based on hybrid neural network according to claim 4 is characterized in that: The feature recalibration submodule includes: a global pooling layer, two fully connected layers, an activation function layer and a Scale layer connected in sequence; the global pooling layer is connected to the convolution kernel splicing layer, and the Scale layer is also connected to the convolution kernel splicing layer; The global pooling layer is used to compress the fused feature map into a feature vector. The two fully connected layers and the activation function layer are used to learn the weight of each channel according to the feature vector to capture the relationship between channels and obtain a weight vector for each channel; The Scale layer is used to apply the weight vector of each channel to each channel in the fused feature map to obtain an optimized feature map.

6. The image recognition method based on hybrid neural network according to claim 1, characterized in that: The hybrid neural network model also includes an input layer, a convolutional layer, a maximum pooling layer, a global pooling layer, a fully connected layer and a softmax layer; The input layer, the convolution layer and the maximum pooling layer are connected in sequence, the maximum pooling layer is connected to the first hybrid neural network module, the last hybrid neural network module is connected to the global pooling layer, and the global pooling layer is connected to the fully connected layer and the softmax layer in sequence; The identifying of the test image based on the trained hybrid neural network model includes: The convolution layer performs feature extraction on the test image to obtain an initial feature map; the maximum pooling layer performs dimensionality reduction processing on the initial feature map to obtain a compressed feature map; wherein the compressed feature map serves as input information of the first hybrid neural network module; Each hybrid neural network module processes the received input information until the last hybrid neural network module outputs a corresponding output feature map, wherein the output feature map of the previous hybrid neural network module serves as input information of the next hybrid neural network module; The global pooling layer performs an average operation on each channel of the received output feature map, calculates the average value of all values ​​on each channel, and obtains a one-dimensional vector; The fully connected layer and the softmax layer output recognition results according to the one-dimensional vector.

7. The image recognition method based on hybrid neural network according to any one of claims 1 to 6, characterized in that: Before training the hybrid neural network model, the method further includes: Xavier is used to initialize the hybrid neural network model.

8. The image recognition method based on hybrid neural network according to any one of claims 1 to 6, characterized in that: The adopting the image data set to train the hybrid neural network model specifically includes: The initialized hybrid neural network model is trained on the training set using the Adam optimizer; The optimized hybrid neural network model is validated on the validation set and hyperparameters are tuned.

9. A device for image recognition based on a hybrid neural network, characterized in that: The device comprises: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the image recognition method based on a hybrid neural network as described in any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the image recognition method based on hybrid neural network as described in any one of claims 1 to 8.

Images