Image recognition method and system based on space-spectrum joint model, and medium

By using the cascaded residual structure and attention mechanism of the space-spectrum joint model, the problem of low accuracy in recognizing multiple target objects with similar colors or colorless and transparent objects is solved, achieving higher recognition accuracy and robustness.

CN118918442AActive Publication Date: 2024-11-08LIAOCHENG UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411069132.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-08
Estimated Expiration
2044-08-06

AI Technical Summary

Technical Problem

Existing image recognition technologies have low accuracy when identifying multiple types of objects with similar colors or colorless and transparent objects. Traditional algorithms have poor robustness, and deep learning algorithms are prone to gradient vanishing or exploding, making it difficult to obtain global information.

Method used

An image recognition method based on a space-spectrum joint model is adopted. By combining the cascaded residual structure of spectral feature extraction network and spatial feature extraction network with high-dimensional mapping, channel attention mechanism and spatial attention mechanism, a multi-class target object recognition method is performed.

Benefits of technology

It improves the accuracy and robustness of multi-target object recognition, effectively captures global information of the target object, avoids gradient vanishing or exploding problems, and improves recognition accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918442A_ABST
    Figure CN118918442A_ABST
Patent Text Reader

Abstract

The invention discloses an image recognition method and system based on a space-spectrum joint model, and a medium, and relates to the field of image recognition, and the method comprises the steps: obtaining a plurality of different sample images; preprocessing all the sample images to obtain a plurality of sample slice images; dividing all sample slice images into a training set, a verification set and a test set in proportion; constructing an image recognition basic model; based on the training set and the label set, training the image recognition basic model until the average accuracy rate and the total accuracy rate of recognition of multiple types of target bodies in the sample slice image are both greater than a set threshold value; testing and verifying the trained image recognition basic model by using the test set and the verification set, and determining the trained, tested and verified image recognition basic model as a space-spectrum joint model; and obtaining a to-be-recognized image, inputting the to-be-recognized image into the space-spectrum joint model, and recognizing to obtain multiple types of target bodies in the to-be-recognized image. According to the invention, the recognition precision of multiple types of target bodies in the image is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an image recognition method, system and medium based on a spatial-spectral joint model. Background Art

[0002] With the continuous advancement of image recognition technology, its application in animal husbandry, agriculture, medicine and industry is becoming more and more extensive. In animal husbandry, the identification of pasture grass can effectively prevent the invasion of foreign grass species, realize the degradation detection of grassland, protect local grass species and provide a good and sufficient source of feed for animal husbandry; in agriculture, image recognition technology provides an efficient and non-destructive detection method for the classification and detection of pathogens and molds and the classification of crop seeds, which provides certain help for the control of pathogens and molds and the correct classification of crop seeds; in medicine, image recognition technology can provide effective technical support for the identification of the authenticity and quality of medical cotton.

[0003] However, image recognition technology is not widely used in the identification and screening of seed cotton impurities in existing cotton processing plants, and in the classification and recycling of transparent and white materials in landfills. This is because, when faced with multiple types of targets with similar colors or colorless and transparent, ordinary visual cameras cannot effectively distinguish between multiple types of targets due to their extremely small color differences. Traditional algorithms, such as Gaussian classifiers and support vector machines, have poor robustness and are not suitable for the identification and classification of multiple types of targets. For example, the convolutional neural network in the deep learning algorithm obtains relatively single scale information, and it is difficult to obtain the global information of multiple types of targets, resulting in a low accuracy rate in identifying multiple types of targets.

[0004] With the development of the times, many scholars have increased the number of layers of convolutional neural networks to obtain a larger receptive field and thus obtain the global information of the target. However, with the surge in the number of network layers, the convolutional neural network will undoubtedly suffer from problems such as gradient vanishing or gradient explosion, which will ultimately lead to a significant reduction in the accuracy of multi-class target recognition in images. Summary of the invention

[0005] The purpose of this application is to provide an image recognition method, system and medium based on a spatial-spectral joint model, which improves the accuracy of identifying multiple types of targets in an image.

[0006] To achieve the above objectives, this application provides the following solutions.

[0007] In the first aspect, the present application provides an image recognition method based on a spatial-spectral joint model, and the image recognition method based on the spatial-spectral joint model includes: obtaining multiple different sample images; each sample image contains multiple categories of target bodies; the target bodies of each category have similar colors or are colorless and transparent; preprocessing all sample images to obtain multiple sample slice images; the preprocessing at least includes: black and white correction, three-dimensional cropping, standardization, multivariate scattering correction and slicing; all sample slice images are divided into a training set, a validation set and a test set in proportion; constructing an image recognition basic model; the image recognition basic model at least includes: a spectral feature extraction network and a spatial feature extraction network; the spectral feature extraction network includes: a first hierarchical residual structure and a second hierarchical residual structure; the spatial feature extraction network includes: a third hierarchical residual structure and a fourth hierarchical residual structure; the first hierarchical residual structure and the second hierarchical residual structure are connected in a cascade manner; the third hierarchical residual structure and the fourth hierarchical residual structure are connected in a cascade manner; the first hierarchical residual structure and the third hierarchical residual structure both contain n Layer structure, n The first layered residual structure and the third layered residual structure both have n -1 layer structure contains convolution blocks; the second layer residual structure and the fourth layer residual structure both contain m Layer structure, m The second layer residual structure and the fourth layer residual structure are both connected in a cascade manner. m -1 layer structure contains convolutional blocks; n , m is an integer greater than 2 and satisfies n > m ; Based on the training set and the label set, the image recognition basic model is trained until the average accuracy and overall accuracy of the recognition of multiple types of target bodies in the sample slice image are greater than the set threshold; the label set contains multiple sample slice images with various types of target bodies labeled; the trained image recognition basic model is tested and verified using the test set and the verification set, and the image recognition basic model after training, testing and verification is determined as a spatial-spectral joint model; an image to be recognized is obtained, and the image to be recognized is input into the spatial-spectral joint model to recognize multiple types of target bodies in the image to be recognized.

[0008] Optionally, the image recognition basic model also includes: a high-dimensional mapping module, a channel attention mechanism module, a spatial attention mechanism module and an information fusion module; the high-dimensional mapping module is connected to the spectral feature extraction network and the spatial feature extraction network respectively; the spectral feature extraction network is connected to the channel attention mechanism module; the spatial feature extraction network is connected to the spatial attention mechanism module; the channel attention mechanism module and the spatial attention mechanism module are connected to the information fusion module respectively; the high-dimensional mapping module is used to increase the dimension of the sample slice image to obtain a high-spectral sample slice image; the spectral feature extraction network is used to extract the The spectral features of the hyperspectral sample slice image are used to obtain an initial spectral feature map; the spatial feature extraction network is used to extract the spatial features of the hyperspectral sample slice image to obtain an initial spatial feature map; the channel attention mechanism module is used to change the channel attention weight of the initial spectral feature map to obtain an optimized spectral feature map; the spatial attention mechanism module is used to change the spatial attention weight of the initial spatial feature map to obtain an optimized spatial feature map; the information fusion module is used to fuse the optimized spectral feature map and the optimized spatial feature map to obtain feature fusion information; the feature fusion information is used to identify multiple types of targets in the sample slice image.

[0009] Optionally, the operation relationship within the first hierarchical residual structure is as follows.

[0010] .

[0011] In the formula, is the first layer of the residual structure i The output of the layer structure is is the first layer of the residual structure n The input of the layer structure is is the first layer of the residual structure n -1 layer structure output, is the first convolution operator.

[0012] Optionally, the operation relationship within the second hierarchical residual structure is as follows.

[0013] .

[0014] In the formula, is the first i The output of the layer structure is is the first m The input of the layer structure is is the first m -1 layer structure output, is the first convolution operator.

[0015] Optionally, the operation relationship within the third hierarchical residual structure is as follows.

[0016] .

[0017] In the formula, is the third layer residual structure i The output of the layer structure is is the third layer residual structure n The input of the layer structure is is the third layer residual structure n -1 layer structure output, is the second convolution operator.

[0018] Optionally, the operation relationship within the fourth hierarchical residual structure is as follows.

[0019] .

[0020] In the formula, is the fourth layer of the residual structure i The output of the layer structure is is the fourth layer of the residual structure m The input of the layer structure is is the fourth layer of the residual structure m -1 layer structure output, is the second convolution operator.

[0021] Optionally, the calculation formula for the average accuracy is as follows.

[0022] .

[0023] In the formula, is the average accuracy, For the correct identification j The number of samples of the target class, For the j The total number of samples of the target class, is the total number of categories of the target object; The calculation formula of the overall accuracy is as follows.

[0024] .

[0025] In the formula, is the overall accuracy, is the total number of samples of all categories of targets, The number of correctly identified samples in all categories of objects.

[0026] In a second aspect, the present application also provides a computer system, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image recognition method based on the spatial-spectral joint model.

[0027] In a third aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image recognition method based on the spatial-spectral joint model.

[0028] According to the specific embodiments provided in this application, this application discloses the following technical effects.

[0029] The present application considers how to improve the accuracy of multi-class target recognition in images from two aspects. First, in the selection of training set images, the sample images selected by the present application have similar colors or are colorless and transparent, and sample images with large color differences are not selected, so as to ensure that the training set images are more targeted. Secondly, the present application also performs pre-processing such as black and white correction, multivariate scattering correction, and slicing on the sample images, further improving the accuracy of the training set images. Second, in terms of model structure, in order to obtain a larger receptive field and avoid problems such as gradient disappearance or explosion, the present application designs a front-to-back cascaded hierarchical residual structure, namely the first hierarchical residual structure and the second hierarchical residual structure in the spectral feature extraction network, and the third hierarchical residual structure and the fourth hierarchical residual structure in the spatial feature extraction network. Because each residual structure is composed of multiple layer structures, a larger receptive field can be obtained. At the same time, in order to avoid too many layers, the multiple layer structures are separated by a front-to-back cascade method. Based on the above two considerations, the present application can not only ensure the effectiveness of model training, but also use the model structure to capture the global information of the target body, thereby improving the accuracy of multi-class target recognition in images. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 A flowchart of an image recognition method based on a spatial-spectral joint model provided in an embodiment of the present application.

[0032] Figure 2 A schematic diagram of the structure of the high-dimensional mapping module-spectral feature extraction network-channel attention mechanism module provided in an embodiment of the present application.

[0033] Figure 3 A schematic diagram of the structure of the high-dimensional mapping module-spatial feature extraction network-spatial attention mechanism module provided in an embodiment of the present application.

[0034] Figure 4 A schematic diagram of the structure of the information fusion module provided in the embodiment of the present application.

[0035] Figure 5 A schematic diagram of the structure of the space-spectrum joint model provided in an embodiment of the present application.

[0036] Figure 6 This is a diagram of the internal structure of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] The purpose of this application is to provide an image recognition method, system and medium based on a spatial-spectral joint model, which improves the accuracy of identifying multiple types of targets in an image.

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0040] like Figure 1 As shown, this embodiment provides an image recognition method based on a spatial-spectral joint model, which is specifically as follows.

[0041] Step S1: Acquire multiple different sample images; each sample image contains multiple categories of target objects; the target objects of each category have similar colors or are colorless and transparent.

[0042] In this embodiment, the sample image obtained is a seed cotton foreign fiber image, which contains multiple types of target objects such as seed cotton fibers, willow catkins, white polypropylene fibers, and transparent ground film. Specifically, the seed cotton foreign fibers to be identified and classified are placed on the plane to be tested in the dark box, and the GaiaField Pro-V10E hyperspectral camera is fixed on the top of the dark box. The lens of the hyperspectral camera is 500 mm away from the plane to be tested. Four 50-watt halogen light sources are used to fill the shooting area with light, and the hyperspectral camera is controlled by a computer to scan and shoot, and a standard whiteboard image (i.e., an image obtained by shooting pure white nylon cloth), a standard blackboard image (i.e., an image obtained by covering the hyperspectral camera cover) and multiple seed cotton foreign fiber images are obtained respectively.

[0043] Step S2: preprocessing all sample images to obtain a plurality of sample slice images; the preprocessing at least includes: black and white correction, three-dimensional cropping, standardization, multivariate scattering correction and slicing.

[0044] In this embodiment, the process of preprocessing the seed cotton foreign fiber image is specifically as follows.

[0045] In the first step, the black-and-white correction calculation formula is used to perform black-and-white correction processing on the seed cotton foreign fiber image to obtain a black-and-white corrected seed cotton foreign fiber image.

[0046] .

[0047] In the formula, To correct the foreign fiber image of seed cotton in black and white, This is the foreign fiber image of seed cotton. is the standard blackboard image, This is a standard whiteboard image.

[0048] In the second step, the black-and-white corrected seed cotton foreign fiber image is three-dimensionally cropped to remove some useless black background and balance the number of positive and negative samples (i.e., balancing the number of pixels contained in seed cotton fibers, various impurity fibers, background, etc.) to obtain a hyperspectral seed cotton foreign fiber image.

[0049] In the third step, the hyperspectral seed cotton foreign fiber image is subjected to standardization processing and multivariate scattering correction processing in turn. The standardization processing is based on the hyperspectral seed cotton foreign fiber image, and the relationship between its mean and standard deviation is used to obtain a standardized seed cotton foreign fiber image. Among them, the calculation formula for the standardization processing is as follows.

[0050] .

[0051] In the formula, To standardize the foreign fiber image of seed cotton, This is a hyperspectral image of foreign fibers in seed cotton. is the mean reflectance of the hyperspectral seed cotton foreign fiber image, is the standard deviation of the hyperspectral seed cotton foreign fiber image reflectance.

[0052] The multivariate scattering correction process is based on the standardized seed cotton foreign fiber image. By calculating its average spectrum and univariate linear regression, the regression constant and regression coefficient are obtained, and then the relative baseline translation and offset between the near-infrared spectra of each pixel point are corrected to obtain the multivariate scattering correction seed cotton foreign fiber image. The calculation formula of the multivariate scattering correction is as follows.

[0053] .

[0054] .

[0055] .

[0056] In the formula, is the average spectral vector of all pixel spectral data in the standardized seed cotton foreign fiber image, The first i Spectral data of each pixel, is the spectral band in the standardized seed cotton foreign fiber image, For the i The regression coefficient under the spectral data of pixel points is For the i The regression constant under the spectral data of pixel points, For the i Multivariate scattering correction of seed cotton foreign matter image based on spectral data of 1 pixel point.

[0057] The fourth step is to use principal component analysis to reduce the dimension of the multivariate scattering corrected seed cotton foreign fiber image. The high-dimensional space features are mapped to the low-dimensional space through linear transformation, while retaining as much original data as possible, the image dimension is reduced, so that the image is compressed to a large extent. The specific process is as follows: first, a sample set is established for the multivariate scattering corrected seed cotton foreign fiber image and it is centralized; then, the covariance matrix of the sample is calculated and the eigenvalue decomposition is performed to extract the largest The eigenvectors corresponding to the eigenvalues ​​are normalized to form an eigenvector matrix; finally, each sample is converted into a new sample and the new sample set is output.

[0058] The fifth step is to slice the multivariate scattering corrected seed cotton foreign fiber image after dimensionality reduction by principal component analysis according to the set size to generate multiple equal-sized three-dimensional image blocks (i.e., seed cotton foreign fiber slice images).

[0059] Step S3: Divide all sample slice images into training set, validation set and test set in proportion.

[0060] In this embodiment, 5000 seed cotton foreign fiber slice images obtained after preprocessing are divided into a training set, a validation set and a test set in a ratio of 6:2:2.

[0061] Step S4: constructing an image recognition basic model; the image recognition basic model at least includes: a spectral feature extraction network and a spatial feature extraction network; the spectral feature extraction network includes: a first hierarchical residual structure and a second hierarchical residual structure; the spatial feature extraction network includes: a third hierarchical residual structure and a fourth hierarchical residual structure; the first hierarchical residual structure and the second hierarchical residual structure are connected in a cascade manner; the third hierarchical residual structure and the fourth hierarchical residual structure are connected in a cascade manner; the first hierarchical residual structure and the third hierarchical residual structure both contain n Layer structure, n The layers are connected in cascade mode; both the first layer residual structure and the third layer residual structure have n -1 layer structure contains convolutional blocks; the second layer residual structure and the fourth layer residual structure both contain m Layer structure, m The layers are connected in cascade mode; the second layer residual structure and the fourth layer residual structure have m -1 layer structure contains convolutional blocks; n , m is an integer greater than 2 and satisfies n > m .

[0062] Furthermore, the basic model of image recognition also includes: a high-dimensional mapping module, a channel attention mechanism module, a spatial attention mechanism module and an information fusion module; the high-dimensional mapping module is respectively connected to the spectral feature extraction network and the spatial feature extraction network; the spectral feature extraction network is connected to the channel attention mechanism module; the spatial feature extraction network is connected to the spatial attention mechanism module; the channel attention mechanism module and the spatial attention mechanism module are respectively connected to the information fusion module.

[0063] The high-dimensional mapping module is used to upgrade the dimension of the sample slice image to obtain a hyperspectral sample slice image; the spectral feature extraction network is used to extract the spectral features of the hyperspectral sample slice image to obtain an initial spectral feature map; the spatial feature extraction network is used to extract the spatial features of the hyperspectral sample slice image to obtain an initial spatial feature map; the channel attention mechanism module is used to change the channel attention weight of the initial spectral feature map to obtain an optimized spectral feature map; the spatial attention mechanism module is used to change the spatial attention weight of the initial spatial feature map to obtain an optimized spatial feature map; the information fusion module is used to fuse the optimized spectral feature map and the optimized spatial feature map to obtain feature fusion information; the feature fusion information is used to identify multiple types of targets in the sample slice image.

[0064] Among them, the first hierarchical residual structure satisfies the following operation relationship.

[0065] .

[0066] In the formula, is the first layer of the residual structure i The output of the layer structure is is the first layer of the residual structure n The input of the layer structure is is the first layer of the residual structure n -1 layer structure output, is the first convolution operator.

[0067] The second hierarchical residual structure satisfies the following operational relationship internally.

[0068] .

[0069] In the formula, is the first i The output of the layer structure is is the first m The input of the layer structure is is the first m -1 layer structure output, is the first convolution operator.

[0070] The third layer residual structure satisfies the following operation relationship internally.

[0071] .

[0072] In the formula, is the third layer residual structure i The output of the layer structure is is the third layer residual structure n The input of the layer structure is is the third layer residual structure n -1 layer structure output, is the second convolution operator.

[0073] The fourth layer residual structure satisfies the following operation relationship internally.

[0074] .

[0075] In the formula, is the fourth layer of the residual structure i The output of the layer structure is is the fourth layer of the residual structure m The input of the layer structure is is the fourth layer of the residual structure m -1 layer structure output, is the second convolution operator.

[0076] In this embodiment, if Figure 2 As shown in Figure 1, the high-dimensional mapping module consists of a 1×1 convolution layer, a batch normalization layer, and an activation function layer. The high-dimensional mapping module can transform the seed cotton foreign fiber slice images in the training set Upgrading the dimension to slice the cotton fiber The spectral features in the image are mapped to a high-dimensional space, which makes it easier for the spectral feature extraction network and the spatial feature extraction network to learn the spectral information.

[0077] The first layered residual structure in the spectral feature extraction network contains 5 layers, 4 of which contain convolution blocks and the number of convolution blocks in each layer is 6; the second layered residual structure contains 3 layers, 2 of which contain convolution blocks and the number of convolution blocks in each layer is 6. Among them, the convolution blocks in the first layered residual structure and the second layered residual structure are composed of 1×1 convolution layers, batch normalization layers and activation function layers. When the hyperspectral seed cotton foreign fiber slice image obtained by the high-dimensional mapping module When inputting the spectral feature extraction network, the input of each layer structure in the first layer residual structure is the number of channels of the hyperspectral seed cotton foreign fiber slice image. , respectively , , , , after the following formula calculation, the output of each layer structure in the first layered residual structure is .

[0078] .

[0079] The first layer residual structure overall output satisfy: ,Will After inputting the second layer residual structure, the input of each layer structure in the second layer residual structure is Number of channels , respectively , , after the following formula calculation, the output of each layer structure in the second hierarchical residual structure is .

[0080] .

[0081] The second layered residual structure overall output satisfy: , That is the initial spectral feature map obtained through the spectral feature extraction network.

[0082] Will The input channel attention mechanism module is passed through the global maximum pooling and global average pooling respectively, and then passed into the 3×3 convolution layer, the activation function layer and the 3×3 convolution layer in sequence to obtain the first weight and the second weight , and then enter after the following formula calculation Sigmoid Activate the function layer to get the optimized spectral feature map .

[0083] .

[0084] .

[0085] .

[0086] .

[0087] In the formula, is the output weight of the channel attention mechanism module, represents element-wise multiplication, for Sigmoid Activation function, and They represent the values ​​obtained by the global maximum pooling and global average pooling of the channel attention mechanism module, respectively. represents the global maximum pooling, Represents global average pooling.

[0088] like Figure 3 As shown in the figure, the third hierarchical residual structure in the spatial feature extraction network contains 5 layers, 4 of which contain convolution blocks and the number of convolution blocks in each layer is 6; the fourth hierarchical residual structure contains 3 layers, 2 of which contain convolution blocks and the number of convolution blocks in each layer is 6. Among them, the convolution blocks in the third hierarchical residual structure and the fourth hierarchical residual structure are composed of 3×3 convolution layers, batch normalization layers and activation function layers. When the hyperspectral seed cotton foreign fiber slice image obtained by the high-dimensional mapping module When inputting the spatial feature extraction network, the input of each layer in the third layer residual structure is the number of channels of the hyperspectral seed cotton foreign fiber slice image. , respectively , , , , after the following formula calculation, the output of each layer structure in the third layered residual structure is .

[0089] .

[0090] The third layer residual structure overall output satisfy: ,Will After inputting the fourth layer residual structure, the input of each layer structure in the fourth layer residual structure is Number of channels , respectively , , after the following formula calculation, the output of each layer structure in the fourth layered residual structure is .

[0091] .

[0092] The fourth layer residual structure overall output satisfy: , That is the initial spatial feature map obtained through the spatial feature extraction network.

[0093] Will The input spatial attention mechanism module is used to concatenate feature maps after global maximum pooling and global average pooling, and then passed to a 3×3 filter for convolution operation. Sigmoid After the activation function and the following formula are calculated, the optimized spatial feature map is obtained .

[0094] .

[0095] .

[0096] .

[0097] .

[0098] In the formula, is the output weight of the spatial attention mechanism module, represents a 3×3 convolution operation, represents feature map concatenation, and They represent the values ​​obtained by the global maximum pooling and global average pooling of the spatial attention mechanism module respectively.

[0099] like Figure 4 As shown, the optimized spectral characteristic diagram and optimize spatial feature maps The information is fused through two layers in the information fusion module (consisting of a batch normalization layer, an activation function layer, and a random inactivation layer), and then input into the fully connected layer to complete the seed cotton foreign fiber slice image. Identification and classification of various types of targets in .

[0100] Step S5: Based on the training set and the label set, the basic model of image recognition is trained until the average accuracy and overall accuracy of multi-class target recognition in the sample slice image are greater than the set threshold; the label set contains multiple sample slice images labeled with various types of target.

[0101] In this embodiment, the label set is determined after preprocessing the seed cotton foreign fiber image, that is, various target bodies to be identified are manually calibrated in the preprocessed seed cotton foreign fiber image, and finally the label set is determined based on multiple seed cotton foreign fiber images with various target bodies calibrated.

[0102] As a preferred implementation method, this embodiment determines the average accuracy and overall accuracy from two aspects. First, the convolution kernel size in the spatial feature extraction network is continuously changed, and then the image recognition basic model is used to perform recognition and classification experiments on multiple types of targets, and the average accuracy and overall accuracy of the recognition and classification results of multiple types of targets are calculated based on the experimental results and the following formula, until the average accuracy and overall accuracy are both greater than 98%, and the optimal convolution kernel size in the spatial feature extraction network is recorded at this time.

[0103] .

[0104] .

[0105] In the formula, is the average accuracy, For the correct identification j The number of samples of the target class, For the j The total number of samples of the target class, is the total number of target categories, is the overall accuracy, is the total number of samples of all categories of targets, The number of correctly identified samples in all categories of objects.

[0106] Secondly, the optimal convolution kernel size in the above-mentioned spatial feature extraction network is kept unchanged, and the slice size of the sample image in the preprocessing process is continuously changed. Then, the basic image recognition model is used to conduct multi-class target recognition and classification experiments. Based on the experimental results and the above formula, the average accuracy and overall accuracy of the multi-class target recognition and classification results are calculated again until the average accuracy and overall accuracy are greater than 98%. The optimal slice size of the sample image in the preprocessing process is recorded at this time.

[0107] Step S6: Use the test set and the validation set to test and validate the trained basic image recognition model, and determine the trained, tested and validated basic image recognition model as a spatial-spectral joint model.

[0108] In this embodiment, based on the optimal convolution kernel size and the optimal slice size determined in step S5, the trained image recognition basic model is tested and verified using the test set and the validation set, and the parameters in the trained image recognition basic model are continuously adjusted until the optimal value is reached, and finally the following is obtained: Figure 5 The joint spatial-spectral model shown.

[0109] Step S7: Obtain the image to be identified, and input the image to be identified into the spatial-spectral joint model to identify multiple types of target objects in the image to be identified.

[0110] In this embodiment, the identified multiple types of targets will be marked in different colors in the image to be identified, which is convenient for operators to observe and classify.

[0111] Based on the above analysis, this application mainly has the following advantages.

[0112] 1) In the spectral feature extraction network and the spatial feature extraction network, each layered residual structure transmits information to the adjacent layer structure through different branches, and finally cascades the information in each layer structure. At the same time, the cascade method of the two layered residual structures and the fusion design of the channel attention mechanism fully learn the spectral feature information.

[0113] 2) The spatial-spectral joint model can obtain a rich receptive field and can fully learn to extract the spectral information and spatial position information of the hyperspectral image. Compared with the traditional recognition method, the feature description and classification recognition results of the target to be identified in this application are more accurate, reasonable and robust.

[0114] In another exemplary embodiment, the present application also provides a computer system, which may be a server or a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer system includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer system is used to provide computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer system is used to store video tag processing data. The input / output interface of the computer system is used to exchange information between the processor and an external device. The communication interface of the computer system is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image recognition method based on a space-spectrum joint model is implemented.

[0115] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer system to which the scheme of the present application is applied. The specific computer system may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] In another exemplary embodiment, the present application further provides a computer-readable storage medium storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0117] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0118] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0119] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0120] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An image recognition method based on a spatial-spectral joint model, It is characterized in that The image recognition method based on the spatial-spectral joint model includes: Acquire multiple different sample images; each sample image contains multiple categories of target objects; the target objects of each category have similar colors or are colorless and transparent; Preprocessing all sample images to obtain multiple sample slice images; the preprocessing at least includes: black and white correction, three-dimensional cropping, standardization, multivariate scattering correction and slicing; Divide all sample slice images into training set, validation set and test set in proportion; Construct an image recognition basic model; the image recognition basic model at least includes: a spectral feature extraction network and a spatial feature extraction network; the spectral feature extraction network includes: a first hierarchical residual structure and a second hierarchical residual structure; the spatial feature extraction network includes: a third hierarchical residual structure and a fourth hierarchical residual structure; the first hierarchical residual structure and the second hierarchical residual structure are connected in a cascade manner; the third hierarchical residual structure and the fourth hierarchical residual structure are connected in a cascade manner; the first hierarchical residual structure and the third hierarchical residual structure both contain n Layer structure, n The first layered residual structure and the third layered residual structure both have n -1 layer structure contains convolution blocks; the second layer residual structure and the fourth layer residual structure both contain m Layer structure, m The second layer residual structure and the fourth layer residual structure are both connected in a cascade manner. m -1 layer structure contains convolutional blocks; n , m is an integer greater than 2 and satisfies n > m ; Based on the training set and the label set, the image recognition basic model is trained until the average accuracy and the overall accuracy of the recognition of multiple types of target bodies in the sample slice image are greater than the set threshold; the label set contains multiple sample slice images labeled with various types of target bodies; Using the test set and the validation set to test and validate the trained image recognition basic model, and determining the trained, tested and validated image recognition basic model as a spatial-spectral joint model; An image to be identified is acquired, and the image to be identified is input into the spatial-spectral joint model to identify multiple types of target objects in the image to be identified.

2. The image recognition method based on the spatial-spectral joint model according to claim 1, It is characterized in that The image recognition basic model also includes: a high-dimensional mapping module, a channel attention mechanism module, a spatial attention mechanism module and an information fusion module; The high-dimensional mapping module is connected to the spectral feature extraction network and the spatial feature extraction network respectively; the spectral feature extraction network is connected to the channel attention mechanism module; the spatial feature extraction network is connected to the spatial attention mechanism module; the channel attention mechanism module and the spatial attention mechanism module are connected to the information fusion module respectively; The high-dimensional mapping module is used to upgrade the dimension of the sample slice image to obtain a hyperspectral sample slice image; the spectral feature extraction network is used to extract the spectral features of the hyperspectral sample slice image to obtain an initial spectral feature map; the spatial feature extraction network is used to extract the spatial features of the hyperspectral sample slice image to obtain an initial spatial feature map; the channel attention mechanism module is used to change the channel attention weight of the initial spectral feature map to obtain an optimized spectral feature map; the spatial attention mechanism module is used to change the spatial attention weight of the initial spatial feature map to obtain an optimized spatial feature map; the information fusion module is used to perform information fusion on the optimized spectral feature map and the optimized spatial feature map to obtain feature fusion information; the feature fusion information is used to identify multiple types of targets in the sample slice image.

3. The image recognition method based on the spatial-spectral joint model according to claim 1, It is characterized in that The operation relationship within the first hierarchical residual structure satisfies: ; In the formula, is the first layer of the residual structure i The output of the layer structure is is the first layer of the residual structure n The input of the layer structure is is the first layer of the residual structure n -1 layer structure output, is the first convolution operator.

4. The image recognition method based on the spatial-spectral joint model according to claim 1, It is characterized in that The operation relationship within the second hierarchical residual structure satisfies: ; In the formula, is the first i The output of the layer structure is is the first m The input of the layer structure is is the first m -1 layer structure output, is the first convolution operator.

5. The image recognition method based on the spatial-spectral joint model according to claim 1, It is characterized in that The operation relationship within the third hierarchical residual structure satisfies: ; In the formula, is the third layer residual structure i The output of the layer structure is is the third layer residual structure n The input of the layer structure is is the third layer residual structure n -1 layer structure output, is the second convolution operator.

6. The image recognition method based on the spatial-spectral joint model according to claim 1, It is characterized in that The operation relationship within the fourth hierarchical residual structure satisfies: ; In the formula, is the fourth layer of the residual structure i The output of the layer structure is is the fourth layer of the residual structure m The input of the layer structure is is the fourth layer of the residual structure m -1 layer structure output, is the second convolution operator.

7. The image recognition method based on the spatial-spectral joint model according to claim 1, It is characterized in that The calculation formula of the average accuracy is: ; In the formula, is the average accuracy, For the correct identification j The number of samples of the target class, For the j The total number of samples of the target class, is the total number of categories of the target object; The calculation formula of the overall accuracy is: ; In the formula, is the overall accuracy, is the total number of samples of all categories of targets, The number of correctly identified samples in all categories of objects.

8. A computer system, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image recognition method based on the spatial-spectral joint model described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the image recognition method based on the spatial-spectral joint model described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Hyperspectral image classification method based on double-branch spectrum multi-scale attention network

    CN113486851A

  • Early warning method and system for vehicle lane departure in night scene

    CN115880658A

  • Hyperspectral image classification method based on spectrum-spatial residual attention mechanism convolutional neural network

    CN118314396A

  • Method for classifying hyperspectral images on basis of adaptive multi-scale feature extraction model

    WO2022160771A1