Discrete object classification method, device, equipment and computer storage medium

Feature extraction and fusion of tea images through capsule network and frequency domain decomposition technology, solving the problem of weak recognition ability in tea classification and achieving higher recognition accuracy and accuracy.

CN114638981BActive Publication Date: 2024-12-03CHINA MOBILE CHENGDU INFORMATION & TELECOMM TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011389534.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-01
Publication Date
2024-12-03
Estimated Expiration
2040-12-01

AI Technical Summary

Technical Problem

When the prior art classifies finished tea with discrete properties in fine-grained classification, there are problems such as weak recognition ability and difficult to guarantee the accuracy rate, especially due to the discrete random distribution of tea and image processing. .

Method used

Capsule network and frequency domain decomposition technology are used to extract and fusion the tea image. Through the fusion of capsule tensor and frequency domain tensor, accurate identification of tea categories is achieved.

Benefits of technology

It improves the accuracy and recognition accuracy of tea classification, reduces misjudgment and misjudgment, and enhances the recognition ability of tea fine-grained classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114638981B_ABST
    Figure CN114638981B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a classification method, device, equipment and computer storage medium for discrete objects. The method comprises collecting a first image of a discrete object, processing the first image respectively by using a capsule network and a frequency domain decomposition method, and fusing the capsule tensor and the frequency domain tensor respectively obtained to obtain tensor fusion data; determining the category of the discrete object corresponding to the first image from a preset category according to the tensor fusion data; the present disclosure obtains a frequency domain tensor of frequency domain feature information in the discrete object image by frequency domain decomposition, and obtains a capsule tensor by extracting basic features in the discrete object image by using a capsule network, and more accurately identifies the discrete object by fusing the two tensors, and then achieves accurate classification, which can effectively solve the problem of weak entity feature recognition capability of the existing convolutional neural network, is suitable for fine-grained classification and recognition of digital images of discrete objects, and ensures recognition accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure belongs to the technical field of discrete object fine-grained classification, and in particular, relates to a discrete object classification method, device, equipment and computer storage medium. Background Art

[0002] At present, it is very difficult to classify discrete objects in fine-grained classification. For example, the classification of finished tea leaves can be divided into many categories according to the tea trees, buds, regions, and tea-making processes. Tea classification is mostly achieved through the recognition and processing of tea images. However, due to the fine-grained classification characteristics, the two main existing classification technologies have some problems in the classification process of finished tea leaves with discrete properties:

[0003] One of the classification techniques is: based on the discrete and indivisible distribution characteristics of finished tea leaves, the entire image is regarded as a statistical distribution problem of color distribution, and the data dimension is increased by modeling the discrete distribution and finally reduced to the number of data categories; the disadvantage of this classification technology is that it requires a lot of effort for data preprocessing, and the accuracy of the recognition results is highly dependent on human ability, and the generalization ability is weak;

[0004] Another method is to use mechanical centrifugal rotation distribution, or collect a specified mass of tea leaves in a uniformly distributed manner to obtain a fixed distribution image, and use a convolutional neural network to recognize the image. This classification scheme has great limitations. It is difficult to apply due to the discrete and random distribution of tea leaves. In addition, this processing method will cause the image to lose a lot of detailed information, so the recognition accuracy is difficult to guarantee when used for tea classification, and it is also very easy to cause missed judgments, misjudgments, and wrong judgments.

[0005] Therefore, the existing technology has the problem of poor classification results for fine-grained classification of discrete objects such as finished tea leaves. Summary of the invention

[0006] The embodiments of the present disclosure provide a method, device, equipment and computer storage medium for classifying discrete objects, which can be applied to the fine-grained classification of discrete objects such as finished tea leaves, and solve the problem that the prior art has weak ability to recognize digital images of discrete objects and difficulty in ensuring accuracy.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for classifying discrete objects, the method comprising:

[0008] Acquiring a first image of a discrete object;

[0009] Using a capsule network, extracting target features from the first image, and outputting a capsule tensor of the first image;

[0010] The capsule tensor is fused with the frequency domain tensor of the first image to obtain tensor fusion data of the first image; wherein the frequency domain tensor is obtained by extracting frequency domain information features of the first image using a frequency domain decomposition method;

[0011] According to the tensor fusion data, a category of the discrete object corresponding to the first image is determined from preset categories.

[0012] In some embodiments, the capsule network includes a convolutional layer, a plurality of main capsule layers, and a digital capsule layer; the capsule network is used to extract target features from the first image, and the capsule tensor of the first image is output, including:

[0013] Inputting the first image into the convolution layer for convolution calculation, and the convolution layer outputs target feature data of the first image;

[0014] Input the target feature data into several main capsule layers accordingly, combine the target feature data through the several main capsule layers, and output the first tensor;

[0015] The first tensor is input into the digital capsule layer. The digital capsule layer has a plurality of digital capsules, and each preset category corresponds to a digital capsule. The plurality of digital capsules all use the first tensor as input data, and are calculated through a dynamic routing algorithm to output a capsule tensor.

[0016] In some embodiments, frequency domain information feature extraction is performed on the first image using a frequency domain decomposition method, including:

[0017] Performing frequency domain decomposition on the digital signal of the first image to obtain a second tensor; wherein the second tensor is data of high-frequency components decomposed from the first image, or the second tensor is data of low-frequency components mixed in the high-frequency components decomposed from the first image;

[0018] Recursively decompose the low-frequency components towards the high-frequency components;

[0019] Perform multiple direction filtering on the second tensor to obtain feature images in each direction;

[0020] Performing Fourier transformation on the feature image to obtain multiple frequency domain transformation images containing high frequency component distribution information;

[0021] Perform convolution pooling on multiple frequency domain transformed images and output frequency domain tensors.

[0022] In some embodiments, the frequency domain tensor has the same dimension as the capsule tensor.

[0023] In some embodiments, convolution pooling is performed on the frequency domain transformed image, and a frequency domain tensor is output; including:

[0024] The frequency domain transformed image is subjected to multiple convolution pooling processes, and the frequency domain transformed image is downsampled to the same output dimension as the capsule network to obtain a frequency domain tensor with the same dimension as the capsule tensor, and the frequency domain tensor is output.

[0025] In some embodiments, fusing the capsule tensor and the frequency domain tensor to obtain tensor fusion data of the first image includes:

[0026] Through one-dimensional convolution, the capsule tensor and the frequency domain tensor are fully connected to obtain a feature fusion tensor;

[0027] The feature fusion tensor is reduced in dimension by one-dimensional convolution according to the number of preset categories to obtain tensor fusion data with the same number of preset categories.

[0028] In some embodiments, after determining the category of the discrete object corresponding to the first image from preset categories according to the tensor fusion data, the method further includes:

[0029] The preset recognition model is trained by the classification method of discrete objects to classify and recognize discrete objects;

[0030] acquiring a second image of the discrete object;

[0031] recognizing the second image by using the recognition model;

[0032] Output the category of the discrete object corresponding to the second image.

[0033] In a second aspect, an embodiment of the present disclosure provides a discrete object classification device, the device comprising:

[0034] A first acquisition module, used for acquiring a first image of a discrete object;

[0035] A capsule network module, used to extract target features from the first image using a capsule network, and output a capsule tensor of the first image;

[0036] A frequency domain decomposition module, used to extract frequency domain information features of the first image by frequency domain decomposition, and output a frequency domain tensor of the first image;

[0037] Fusion module: used to fuse the capsule tensor and the frequency domain tensor to obtain tensor fusion data of the first image;

[0038] Classification module: used to determine the category of the discrete object corresponding to the first image from preset categories based on the tensor fusion data.

[0039] In some embodiments, the apparatus further comprises:

[0040] A recognition model is used to classify and recognize discrete objects; the recognition model is obtained by training the first acquisition module, the capsule network module, the frequency domain decomposition module, the fusion module and the class module;

[0041] A second acquisition module, used for acquiring a second image of the discrete object;

[0042] The output module is used to output the category information of the discrete object corresponding to the second image after the second image is recognized by the recognition model.

[0043] In a third aspect, an embodiment of the present disclosure provides a discrete object classification device, the device comprising:

[0044] A processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the discrete object classification method as described in any of the above embodiments.

[0045] In a fourth aspect, an embodiment of the present disclosure provides a computer storage medium, characterized in that computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, a discrete object classification method as in any of the above embodiments is implemented.

[0046] The discrete object classification method, device, equipment and computer storage medium of the embodiments of the present invention obtain a frequency domain tensor of frequency domain feature information in a discrete object image through frequency domain decomposition, and extract basic features in the discrete object image through a capsule network to obtain a capsule tensor, and by fusing the two tensors, more accurately identify the discrete object and then achieve accurate classification, which can effectively solve the problem of weak entity feature recognition capability of existing convolutional neural networks, and is suitable for fine-grained classification and recognition of digital images of discrete objects (such as finished tea leaves) to ensure recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiment of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 is a flow chart of a method for classifying discrete objects provided by an embodiment of the present disclosure;

[0049] Figure 2 is a flow chart of a method for classifying discrete objects provided by another embodiment of the present disclosure;

[0050] Figure 3 is a schematic diagram of a process of extracting basic features through a capsule network in a discrete object classification method provided by an embodiment of the present disclosure; wherein Figure 3 a is a flowchart of basic feature extraction; Figure 3 b is Figure 3 The data processing diagram of the capsule network when extracting basic features as shown in a;

[0051] Figure 4 It is a flow chart of extracting frequency domain feature information by using frequency domain decomposition in a discrete object classification method provided by an embodiment of the present disclosure;

[0052] Figure 5 yes Figure 4 A schematic diagram of the data processing flow of the frequency domain decomposition method in the process shown;

[0053] Figure 6 is a schematic diagram of a flow chart of fusing two tensors in a discrete object classification method provided by an embodiment of the present disclosure;

[0054] Figure 7 yes Figure 6 Schematic diagram of the convolution processing of tensor fusion in the process shown;

[0055] Figure 8 It is a schematic diagram of the recognition process of an image by a recognition model in a discrete object classification method provided by an embodiment of the present disclosure;

[0056] Fig. 9 is a structural schematic diagram of a discrete object classification device provided by an embodiment of the present disclosure;

[0057] Fig.10 It is a structural schematic diagram of a discrete object classification device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] The features and exemplary embodiments of various aspects of the present disclosure will be described in detail below. In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present disclosure, rather than to limit the present disclosure. For those skilled in the art, the present disclosure can be implemented without the need for some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present disclosure by illustrating examples of the present disclosure.

[0059] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0060] Discrete objects refer to objects whose names correspond to many discrete entities with inconsistent contours. For example, finished tea leaves usually appear as a pile of tea leaves. Although the entities in such discrete objects have commonalities in the same category, the differences between the entities are large or difficult to define uniformly. One of the main characteristics of fine-grained classification is that the differences between classification targets are very small. Therefore, it is very difficult to perform fine-grained classification of such discrete objects through digital images.

[0061] For example, for the fine-grained classification of finished tea, finished tea generally refers to the leaves of evergreen shrub tea trees that can be used to make tea. The drinking method is mostly to brew with boiling water, roll and dry the finished product; there are many types of finished tea. If based on the variety, production method and product appearance, it can be divided into six categories: green tea, black tea, yellow tea, dark tea, oolong tea, and white tea; if based on the season, it can be divided into four categories: spring tea, summer tea, autumn tea, and winter tea; if it is processed from various raw teas or refined tea leaves to form re-added tea, it can be divided into categories such as scented tea, compressed tea, extracted tea, medicinal health tea, tea food, and tea-containing beverages. There are differences in the leaf contour, texture, color, etc. of finished tea between different categories and in the same category. Therefore, it is difficult to ensure the accuracy of fine-grained classification of finished tea by simple color statistics methods or convolutional neural network recognition methods.

[0062] For example, in the prior art, when classifying tea leaves by identifying their color features, the entire digital image of tea leaves is usually considered as a statistical distribution problem of color distribution. However, since the simple statistical distribution of color requires a lot of effort for data preprocessing, the image without obvious feature distinction is adjusted to be more obvious by color balancing, histogram balancing, feature transformation, etc., so that machine learning can be used for better classification and recognition in the data modeling stage. In this way, the strength of human participation seriously affects the final recognition accuracy of the model. In addition, the generalization ability of the model established in this way is weak. When different camera equipment and different camera environments are used, the final color restoration will make the model invalid.

[0063] However, if we use convolutional neural networks, which have excellent performance on large-scale image datasets, to process tea images, there are also many disadvantages. Convolutional Neural Networks (CNN) are a type of feedforward neural networks that include convolution calculations and have a deep structure. They are one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities and can perform a certain degree of shift-invariant classification on input information according to their hierarchical structure. Therefore, they are also called "shift-invariant artificial neural networks", that is, they only recognize a certain feature, but do not recognize changes in the position or direction of the feature. Due to the huge difference between tea data and large-scale data sets, the migration effect is not ideal; the shortcomings of this solution are mainly caused by convolutional neural networks: first, convolutional neural networks have certain rotation, translation, and scaling invariance, but they are very limited. They need to collect and train the contour features of the object fixed in advance, and due to the discrete and random distribution of tea and the lack of fixed contours, convolutional neural networks are difficult to apply to the classification of tea; second, convolutional neural networks usually use downsampling for dimensionality reduction, which causes the image to lose a lot of detail information, especially when the differences between tea categories are small and the differences within the category are large, which makes it difficult to guarantee the accuracy of the output; third, convolutional neural networks use feature combination voting for identification, ignoring feature correlation, especially the fresh leaves of finished tea have the same source and similar production process, so it is very easy to cause missed judgments, misjudgments, and wrong judgments.

[0064] In order to solve the problems of the prior art, the embodiments of the present disclosure provide a discrete object classification method, device, equipment and computer storage medium, which are suitable for fine-grained classification and identification of objects with similar discrete characteristics to finished tea.

[0065] The following first introduces the discrete object classification method provided by the embodiment of the present disclosure.

[0066] Figure 1 FIG. 1 is a flow chart of a method for classifying discrete objects provided by an embodiment of the present disclosure. Figure 1 As shown, the method may include the following steps:

[0067] S100. Acquire a first image of a discrete object;

[0068] S200. Using a capsule network, extracting target features from the first image, and outputting a capsule tensor of the first image;

[0069] S300. Using frequency domain decomposition, extract frequency domain information features from the first image, and output a frequency domain tensor of the first image;

[0070] S400. Fusing the capsule tensor with the frequency domain tensor of the first image to obtain tensor fusion data of the first image;

[0071] S500. Determine, based on the tensor fusion data, from preset categories, the category of the discrete object corresponding to the first image.

[0072] In the present disclosure, step S100 captures a first image of a discrete object. The first image may be a training sample for training a recognition model. When used as a training sample, the first image is processed through steps S100 to S500 to train the constructed recognition model. The first image may also be an image to be recognized. When used as an image to be recognized, the first image is processed through steps S100 to S500 and is recognized and classified in the trained recognition model.

[0073] In this embodiment, the discrete object is taken as an example of finished tea leaves to illustrate the method of this embodiment, wherein the collected first image is used as a training sample, and the constructed recognition model is trained through S100 to S500 to realize the classification of discrete objects.

[0074] In a specific embodiment, Figure 2 As shown, step S100 includes S101 . capturing a first image.

[0075] The first image can be collected by using some different brands of smartphones or cameras and other shooting devices, and photographing and collecting six types of tea, namely green tea, black tea, yellow tea, dark tea, oolong tea, and white tea. In this example, in order to improve the generality of the trained recognition model and enhance the diversity of training samples, the smartphone can select several brands of mobile phones such as Xiaomi, Huawei, OPPO, and Apple on the market, and use mobile phones of various brands and styles to collect data on the three types of tea that are selling well in the market. The smartphone camera mode selects the default mode to avoid the filter and other functions of the shooting device from affecting the training effect of the recognition model. Place an unquantified amount of tea on white paper, select a suitable height to focus and take pictures clearly, and the number of images of each type is greater than 400 to form enough training samples.

[0076] Step S100. may also include S102. data cleaning.

[0077] Data cleaning can be performed by manually previewing the collected training samples, deleting repeated, blurred, severely occluded, or severely exposed images, and replacing them with new images to reduce the impact of invalid training samples on the training results.

[0078] Step S100 may also include S103 . data processing.

[0079] During the data processing process, since the finished tea leaves are small objects and are not suitable for non-proportional scaling, data enhancement methods such as image translation, rotation, proportional zooming (15% scale), random hole cropping, etc. can be used in the data processing stage to finally crop the training sample data to a 224*224 image size.

[0080] Step S100 may also include S104: data normalization.

[0081] The sample data after data processing in S103 can be normalized to RGB images, which can speed up the training process and eliminate the effects of lighting changes or shadows in some RGB color processing stages. The normalization formula is as follows:

[0082]

[0083] R, G, and B are three color channels in conventional image processing, and (r, g, b) represents the pixel value of a certain pixel of the image.

[0084] The image acquired in step S100 can be processed in steps S200 and S300 to extract different feature information.

[0085] Wherein step S200 is as follows Figure 3 In Figure 3 As shown in a, a capsule network is used to extract target features of the first image and output the capsule tensor of the first image.

[0086] Capsule Network (CapsNet) is a neural network architecture that includes a three-layer encoder structure of a convolutional layer, several main capsule layers, and a digital capsule layer. It can also include multiple layers of fully connected layers (not shown in the figure). In the capsule network, the main capsule layer and the digital capsule layer are composed of capsule units. A capsule unit is a group of neurons that can learn to detect a specific object in a given area and output a vector. The length of the vector represents the probability of the occurrence of a specific object, and the direction of the vector encodes the posture of the specific object (such as position, orientation). If the specific object is slightly transformed, such as moved, rotated, resized, etc., the capsule will output a vector of the same length, but with a slightly different direction. Therefore, the capsule is equivariant.

[0087] In this embodiment, the basic features of discrete objects on the first image can be marked by the capsule units of the capsule network. For example, for the image of finished tea, the target features may include basic features such as position, size, direction, speed, hue, texture, etc. By extracting and marking these basic signs and using them for training the recognition model, the trained model can have strong generalization capabilities.

[0088] Combination Figure 3 In Figure 3 As shown in b, step S200 may include S201. Crop the first image 301.

[0089] In step S201, the first image 301 obtained by processing in step S100 is first subjected to a center cropping operation. A preferred example given in this embodiment is to crop the first image 301 into a cropped image 302 of a size of 28*28 as input data of the capsule network.

[0090] Step S200 also includes S202. Inputting the first image 301 into the convolution layer 303 for convolution calculation, and the convolution layer 303 outputs the target feature data of the first image.

[0091] Using convolution calculation as a preliminary feature extraction method, the basic features of a 2D image (i.e., a two-dimensional image) can be detected. In this example, the cropped image 302 obtained by cropping in step S201 is input into the convolution layer 303 of the capsule network to extract and calculate the basic feature information of the image. The convolution layer 303 can use 256 9*9*1 convolution kernels to convolve the first image, and use the ReLU activation function y=max(0,x) for nonlinear activation; where y is the activation value and x is the activated value, which can speed up the training speed. The convolution layer in this example can output a 20*20*256 tensor, which is the data of the basic features of the extracted first image, i.e., the target feature data.

[0092] Step S200 also includes S203. Inputting the target feature data into a plurality of main capsule layers 304 accordingly, combining the target feature data through the plurality of main capsule layers 304, and outputting a first tensor.

[0093] In this embodiment, the main capsule layer 304 includes 32 capsule units, which can receive the data of each basic feature of the image extracted by the convolution layer 303, and combine the basic feature data through the 32 capsule units to generate a combination of image features. In this example, the 20*20*256 tensor output by the convolution layer 303 can eventually generate a 6*6*8*32 tensor in the main capsule layer 304, that is, the first tensor, which is the data of the image feature combination.

[0094] Step S200 also includes S204. Inputting the first tensor into the digital capsule layer 305, a plurality of digital capsules all use the first tensor as input data, and perform calculations through a dynamic routing algorithm, and output a capsule tensor 306.

[0095] The digital capsule layer 305 has several digital capsules, each of which is also a capsule unit. Multiple categories can be pre-set for finished tea leaves, and each preset category corresponds to a digital capsule; inside each digital capsule, a weight matrix can be set according to the category, and the mapping calculation between input data and output data can be realized through the weight matrix. In an example of this embodiment, the category of finished tea leaves can be set to 18 categories, and the first tensor of 6*6*8*32 output by the main capsule layer 304 is used as input by each digital capsule, and a 16*18 tensor can be finally output using the dynamic routing algorithm. Among them, the first tensor of 6*6*8*32 can be regarded as an 8-dimensional vector of 6*6*32. Inside the digital capsule, each input tensor maps the 8-dimensional input space to the 16-dimensional capsule output space through the 8*16 weight matrix, and outputs a capsule tensor 306 of a 16*18 matrix.

[0096] Among them, the dynamic routing algorithm is a mature algorithm in this field, which is used to perform propagation updates of capsule networks. Each capsule unit in the previous layer of the capsule network will be connected to each capsule neural unit in the next layer. In the process of connecting and propagating the capsule units, weights and coupling coefficients need to be used for calculation. During the calculation, some common settings of the dynamic routing algorithm include: initializing the bias term to 0, and using the softmax function to determine the coupling coefficient. The principle of the dynamic routing algorithm is that if the tensor directions between two connected capsule units are similar, the dot product operation output of the tensor is a positive number, indicating that the coupling coefficient should become larger, that is, the degree of feature coupling between the two capsule units becomes larger, and the degree of coupling is the degree of consistency from the capsule unit to the next capsule unit. Repeating the iteration of the dynamic routing algorithm can eventually determine a path in a high-dimensional space, that is, a calculation path with high correlation of feature combinations. In this way, in the fine-grained classification process of finished tea, the correlation between basic features can be fully trained, so that even if the finished tea with the same source of fresh leaves or similar production processes in the basic features can be based on the combination with other basic features, accurate classification results can be obtained, thereby improving the neural network's ability to identify entity feature combination paths.

[0097] Since the capsule network can output vectors of the same length for the movement, rotation, and resizing of the basic features of an object, in this embodiment, the capsule network's sensitivity to direction is utilized to perform capsule network processing on the first image, which can effectively mark the rotation, translation, and scaling of the finished tea features, so that the trained recognition model can have better generalization for translation, rotation, and multiple scales.

[0098] In this embodiment, the classification method of discrete objects further includes step S300, and there may be no definite order relationship between step S300 and step S200. Step S200 can extract the basic features of the first image while retaining the correlation between the features in the output capsule tensor, and step S300 can extract the frequency domain information features of the first image layer by layer to avoid the loss of frequency domain information during the training process, which leads to the failure of fine-grained classification of objects with large intra-class differences.

[0099] like Figure 4 As shown, step S300 includes S301. performing frequency domain decomposition on the digital signal of the first image to obtain a second tensor.

[0100] The essence of frequency domain decomposition is signal filtering, refer to Figure 5As shown, the digital signal of the input first image 501 is first decomposed using the Laplace pyramid. The decomposition stage includes two filters, one is a low-pass filter, which is used to obtain the low-frequency component 503 in the image, and the other is a high-pass filter, which is used to obtain the high-frequency component 502. If the low-frequency component is continuously recursively decomposed, more and more detailed image information can be gradually obtained, thereby obtaining frequency domain information at different levels.

[0101] Still taking the classification and recognition of finished tea leaves as an example, the high-frequency component 502, as the segmentation component between the entity and the environment in the image, can gradually reflect the subtle texture features of the finished tea leaves in the image as the recursion deepens. The low-frequency component 503, as the main component of the entity or environment, can gradually provide high-level semantic information to the high-frequency component as the recursion deepens.

[0102] Therefore, the second tensor is data of the high-frequency component 502 decomposed from the first image 501 , or the second tensor is data of the low-frequency component 503 mixed in the high-frequency component 502 decomposed from the first image 501 .

[0103] Step S300 may further include S302: recursively decomposing the low-frequency component 503 toward the high-frequency component 502.

[0104] The first image 501 is decomposed layer by layer, information at different levels is extracted, and input to the high-frequency components. Each recursive processing is observed once, and the result of the decomposition of the low-frequency components can be continuously returned to the high-frequency components for more detailed processing, because the high-frequency components in the image represent information such as the texture or contour of the image. The recursive processing can make the high-frequency components in the second vector more and more, and the extracted entity feature information will be more complete and accurate.

[0105] Step S300 further includes S303. performing multiple direction filtering on the second tensor to obtain feature images 504 in various directions;

[0106] As a preferred example, the present disclosure can filter the second tensor in eight directions through an eight-directional filter to obtain a feature image 504 in each direction. Each feature image 504 only represents information in one direction, so that the high frequency band of the first image can be observed from eight directions, and the differences in different directions can be obtained from various angles. This is conducive to fine-grained classification and recognition during the training process.

[0107] Step S300 further includes S304. Performing Fourier transform on the feature image 504 to obtain a plurality of frequency domain transformed images 505 containing high frequency component distribution information.

[0108] In the feature image 504 in each direction, the high-frequency component area obtains texture or contour information. After the feature image is subjected to Fourier (FT) transform, the high-frequency components are concentrated in the central area, and the low-frequency components are scattered in other areas of the image. Subsequent convolution processing can be regarded as discarding the statistical processing of the low frequency and retaining and balancing the statistical part of the high frequency.

[0109] When a traditional convolutional network is trained, its filter operator only extracts the contour and cannot extract the distribution of the image. Therefore, compared with traditional technology, in the method disclosed in this disclosure, after the FT transformation, the high-frequency components are concentrated and the low-frequency components are dispersed, and the image is distributed twice. This is then connected to the convolution pooling process, which can partially solve the problem of high-dimensional unexplainability of the deep convolution layer (that is, the accuracy of deep learning is unpredictable). Compared with traditional technology, the method disclosed in this disclosure does not need to strictly distribute the image uniformly according to the training requirements, and can also better perform classification and recognition. The method disclosed in this disclosure adds the extraction of image statistical information to the traditional extraction of image semantic information. This multi-responsibility mode can better perform recognition, especially the cluster classification and recognition of discrete objects, and solve the problem of fine-grained classification of discrete objects that cannot be solved by traditional frequency domain decomposition, machine learning, or convolutional neural network alone.

[0110] Therefore, step S300 also includes S305 . Perform convolution pooling processing on multiple frequency domain transformed images 505 , and output a frequency domain tensor 506 .

[0111] In this embodiment, the obtained multiple frequency domain transformation images 505 are convoluted and pooled three times, and the size of the frequency domain transformation image is downsampled to the output dimension of the capsule network. A 3*3 convolution kernel can be used, and the padding attribute (padding) is 1 to keep the input and output dimensions consistent. The pooling uses a 2*2 kernel, and each layer reduces the size of the feature image by half. In this way, a frequency domain tensor 506 with the same dimension as the capsule tensor 306 can be obtained for subsequent tensor data fusion processing.

[0112] Step S300 also includes S306. Decomposing the frequency domain tensor by one-dimensional convolution.

[0113] Since the more convolution pooling layers there are, the more data volume is obtained. After processing the sample data in step S305, the actual amount of tensor data obtained is also large. For this reason, this embodiment uses one-dimensional convolution to directly fuse the frequency domain tensor after convolution pooling processing through S306, thereby reducing the amount of data and reducing training parameters. In the subsequent cross-layer connection process, training can be accelerated.

[0114] The first image is processed through the above steps S200 and S300 to obtain a capsule tensor and a frequency domain tensor respectively, and then step S400 can be performed to fuse the capsule tensor with the frequency domain tensor of the first image to obtain tensor fusion data of the first image.

[0115] Specifically, Figure 6 As shown, step S400 includes S401. Through a one-dimensional convolution method, the capsule tensor and the frequency domain tensor are fully connected to combine features to obtain a feature fusion tensor.

[0116] In this step, through the fully connected layer, reference Figure 7 , using one-dimensional convolution to achieve equivalent mapping of capsule tensor and frequency domain tensor to fully connected output results. Different from traditional neural networks, the disclosed embodiment abandons the softmax layer of the neural network, uses one-dimensional convolution in the fully connected layer, and uses matrix acceleration operations to speed up training and has stronger scalability.

[0117] Step S400 includes S402. The feature fusion tensor is reduced in dimension by one-dimensional convolution according to the number of preset categories to obtain tensor fusion data with the same number of preset categories.

[0118] The capsule tensor and the frequency domain tensor are combined by one-dimensional convolution to perform full-connection feature combination, and then the dimension is reduced to the same number as the preset number of categories by one-dimensional convolution. For example, in the example given in this embodiment, the finished tea is classified into 18 categories, and the tensor fusion data obtained after one-dimensional convolution processing is 18*1.

[0119] S500. Determine, based on the tensor fusion data, from preset categories, the category of the discrete object corresponding to the first image.

[0120] The tensor fusion data is a combination of the basic features and frequency domain features of the first image in different dimensions. In the process of training the recognition model, step S500 of this embodiment continuously trains the weights of the tensor fusion data through a conventional loss function until the corresponding category is correct or infinitely close to the correct one, and the training of the recognition model is completed; then it can be directly used in the classification and recognition of discrete objects.

[0121] For the trained recognition model, in the classification and recognition process, the tensor fusion data already has a fixed weight and can correspond to the preset category. Therefore, it is only necessary to perform S100 to S500 on the acquired image without performing loss function correction on the tensor fusion data. It can correspond to the preset category and complete the recognition process.

[0122] Therefore, in this embodiment, the classification method of discrete objects may further include S600 after S500. The preset recognition model is trained by the classification method of discrete objects to classify and recognize discrete objects. That is, if the preset recognition model is not well trained, the first image is a training sample, and the above step S600 is performed, and the preset recognition model is trained by the method of the above steps S100 to S500. If the preset recognition model is well trained, the first image can be the image to be recognized, and the first image is classified and recognized by the method of the above steps S100 to S500.

[0123] The preset recognition model can be used to build the network using open source deep learning tools (such as PyTorch and TensorFlow).

[0124] When the recognition model is a trained model, such as Figure 8 As shown, after step S600 of this embodiment, classifying and identifying discrete objects may include:

[0125] S601. Collect a second image of the discrete object; the second image is the image to be recognized. The user takes a photo of the discrete object and uploads the photo to the recognition model. The recognition model decodes and compresses the photo, and then performs the recognition process of step S602.

[0126] S602. Identify the second image by using the recognition model; the recognition process includes steps S100 to S500.

[0127] S603. Output the category of the discrete object corresponding to the second image. At the same time, this step can also output the recognition accuracy probability of each category, and arrange them from high to low to verify or analyze the recognition effect of the recognition model.

[0128] The discrete object classification method disclosed in the present invention uses capsule network and frequency domain analysis image processing and data fusion to train the recognition model, which can effectively solve the problem that the convolutional neural network has weak recognition ability for the entity feature combination path, and converts the contour information of the discrete object into statistical graph information through frequency domain decomposition processing, and performs distribution statistics through convolution pooling, and fuses the remaining capsule tensors, which effectively guarantees the accuracy of deep learning; and due to the fused tensor, on the basis of extracting the basic features of the discrete object image, the refined contour information (high-frequency components) of the image is also extracted, so that for the analysis of finished tea with a small amount of impurities, especially for a small amount of other categories of impurities, it can have a higher recognition ability.

[0129] Fig. 9 Schematic diagram of a device structure for discrete objects provided by an embodiment of the present disclosure. Fig. 9 As shown, the device may include

[0130] A first acquisition module 901, used to acquire a first image of a discrete object;

[0131] A capsule network module 902 is used to extract target features from the first image using a capsule network and output a capsule tensor of the first image;

[0132] The frequency domain decomposition module 903 is used to extract frequency domain information features of the first image by frequency domain decomposition, and output a frequency domain tensor of the first image;

[0133] Fusion module 904: used to fuse the capsule tensor and the frequency domain tensor to obtain tensor fusion data of the first image;

[0134] Classification module 905: used to determine the category of the discrete object corresponding to the first image from preset categories according to the tensor fusion data.

[0135] The first acquisition module 901 can be used to implement Figure 1 In step S100 of the discrete object classification method shown in FIG. 1 , a first image is collected as a training sample.

[0136] In another embodiment, the device may further include a preprocessing module 910, the preprocessing module 910 includes a data cleaning module 912, a data processing module 903 and a data normalization module 914, which is used to preprocess the collected first image. Figure 2 In the steps S102 to S104 of the discrete object classification method shown in FIG, data cleaning, data processing and data normalization are performed to improve the data quality of the training samples.

[0137] The first image can be collected by using several brands of mobile phones such as Xiaomi, Huawei, OPPO, and Apple. The data of three types of tea with better sales in the market can be collected by using mobile phones of various brands and styles. The default mode is selected for the camera mode of the smart phone to avoid the filter and other functions of the shooting equipment from affecting the training effect of the recognition model. Place an indefinite amount of tea on a piece of white paper, select a suitable height to focus clearly and take pictures, and the number of images of each type is greater than 400 to form enough training samples. After the first acquisition module 901 obtains these training samples, the samples can be input into the preprocessing module 910 for:

[0138] Data cleaning:

[0139] The data cleaning module 912 can clean the collected training samples through manual preview mode, delete the pictures that are repeatedly taken, blurred, severely blocked, and severely exposed, and take new pictures to replace them, so as to reduce the influence of invalid training samples on the training results;

[0140] Data processing:

[0141] In the data processing process, since the finished tea leaves are small objects and are not suitable for non-proportional scaling, the data processing module 913 can use data enhancement methods such as image translation, rotation, proportional scaling (15% scale), random hole cropping, etc. to finally crop the training sample data into a 224*224 image size;

[0142] Data normalization:

[0143] After data processing, the data normalization module 914 can perform RGB image normalization, which can speed up the training process and eliminate the effects of illumination changes or shadows in some RGB color processing stages. The normalization formula is as follows:

[0144]

[0145] R, G, and B are three color channels in conventional image processing, and (r, g, b) represents the pixel value of a certain pixel of the image.

[0146] The first image processed by the preprocessing module 910 is input into the capsule network module 902 and the frequency domain decomposition module 903 to extract basic features and frequency domain features respectively.

[0147] Among them, the capsule network module 902 can be used to implement Figure 1 or Figure 2 In step S200 of the discrete object classification method shown in FIG, target features are extracted from the first image, and a capsule tensor of the first image is output. Specifically, the capsule network module 902 mainly uses the capsule network architecture to achieve Figure 3 middle Figure 3 In steps S201 to S204 of the discrete object classification method shown in a, the capsule network (CapsNet) is a neural network architecture, including a three-layer encoder structure of a convolutional layer, several main capsule layers, and a digital capsule layer, and may also include multiple layers of fully connected layers.

[0148] The capsule network module 902 can mark the basic features of discrete objects on the first image through the capsule units of the capsule network. For example, for the image of finished tea, the target features may include basic features such as position, size, direction, speed, hue, texture, etc. By extracting and marking these basic signs and using them for training the recognition model, the trained model can have strong generalization ability.

[0149] You can refer to Figure 3 As shown in FIG. 2 , the capsule network module 902 first crops the first image to a size of 28*28 as input data of the capsule network.

[0150] The capsule network module 902 inputs the first image into the convolution layer for convolution calculation, and the convolution layer outputs the target feature data of the first image. In this example, the convolution layer can use 256 9*9*1 convolution kernels to convolve the first image, and use the ReLU activation function y=max(0,x) for nonlinear activation; where y is the activation value and x is the activated value, which can speed up the training. The convolution layer in this example can output a 20*20*256 tensor, which is the data of the basic features of the extracted first image, that is, the target feature data.

[0151] The capsule network module 902 also inputs the target feature data into several main capsule layers, combines the target feature data through several main capsule layers, and outputs a first tensor. In this embodiment, the main capsule layer includes 32 capsule units, which can receive the data of each basic feature of the image extracted by the convolution layer, and combine each basic feature data through 32 capsule units to generate a combination of image features. In this example, the 20*20*256 tensor output by the convolution layer can eventually generate a 6*6*8*32 tensor in the main capsule layer, that is, the first tensor, which is the data of the image feature combination.

[0152] The capsule network module 902 also inputs the first tensor into the digital capsule layer. Several digital capsules use the first tensor as input data, and calculate through the dynamic routing algorithm to output the capsule tensor. The digital capsule layer has several digital capsules, each of which is also a capsule unit. Multiple categories can be pre-set for finished tea leaves, and each preset category corresponds to a digital capsule; inside each digital capsule, a weight matrix can be set according to the category, and the mapping calculation between input data and output data can be realized through the weight matrix. In an example of this embodiment, the category of finished tea leaves can be set to 18 categories, and the first tensor of 6*6*8*32 output by the main capsule layer is used as input by each digital capsule, and a 16*18 tensor can be finally output using the dynamic routing algorithm. Among them, the first tensor of 6*6*8*32 can be regarded as an 8-dimensional vector of 6*6*32. Inside the digital capsule, each input tensor maps the 8-dimensional input space to the 16-dimensional capsule output space through the 8*16 weight matrix, and outputs a capsule tensor of a 16*18 matrix.

[0153] Among them, the dynamic routing algorithm is a mature algorithm in this field. It can determine a path in high-dimensional space, that is, a calculation path with high correlation of feature combinations, by repeating the iteration of the dynamic routing algorithm. In this way, in the fine-grained classification process of finished tea, the correlation between basic features can be fully trained, so that even if the finished tea has the same source of fresh leaves or similar production process in the basic features, accurate classification results can be obtained based on the combination with other basic features, thereby improving the neural network's ability to recognize entity feature combination paths.

[0154] Since the capsule network can output vectors of the same length for the movement, rotation, and resizing of the basic features of an object, in this embodiment, the capsule network's sensitivity to direction is utilized to perform capsule network processing on the first image, which can effectively mark the rotation, translation, and scaling of the finished tea features, so that the trained recognition model can have better generalization for translation, rotation, and multiple scales.

[0155] In this embodiment, the classification device for discrete objects can extract the basic features of the first image through the capsule network module 902, while retaining the correlation between the features in the output capsule tensor, and can also extract the frequency domain information features of the first image layer by layer through the frequency domain decomposition module 903 to avoid the loss of frequency domain information during the training process, resulting in the inability to achieve fine-grained classification of objects with large intra-class differences.

[0156] The frequency domain decomposition module 903 can be used to implement Figure 1 In the step S300 of the discrete object classification method shown in FIG, the digital signal of the first image is decomposed in the frequency domain to obtain a second tensor. Specifically, the frequency domain decomposition module mainly implements the frequency domain decomposition method. Figure 3 Steps S301 to S306 of the discrete object classification method shown in FIG.

[0157] The frequency domain decomposition module 903 first decomposes the digital signal of the input first image using the Laplace pyramid. The decomposition stage includes two filters, one is a low-pass filter, which is used to obtain the low-frequency components in the image, and the other is a high-pass filter, which is used to obtain the high-frequency components. If the low-frequency components are continuously recursively decomposed, more and more detailed image information can be gradually obtained, thereby obtaining frequency domain information at different levels.

[0158] Taking the classification and recognition of finished tea leaves as an example, high-frequency components, as the segmentation components of entities and the environment in the image, can gradually reflect the subtle texture features of the finished tea leaves in the image as the recursion deepens. Low-frequency components, as the main components of the entity or environment, can gradually provide high-level semantic information to the high-frequency components as the recursion deepens.

[0159] Therefore, the second tensor is data of high-frequency components decomposed from the first image, or the second tensor is data of high-frequency components decomposed from the first image mixed with low-frequency components.

[0160] The frequency domain decomposition module 903 may also recursively decompose the low frequency components toward the high frequency components.

[0161] Decompose the first image layer by layer, extract information at different levels, and input it into the high-frequency components. Each recursive process makes an observation, and the results of the decomposition of the low-frequency components can be continuously returned to the high-frequency components for more detailed processing, because the high-frequency components in the image represent information such as the texture or contour of the image. Recursive processing can make the high-frequency components in the second vector more and more, and the extracted entity feature information will be more complete and accurate.

[0162] The frequency domain decomposition module 903 may also perform multiple direction filtering on the second tensor to obtain feature images in various directions;

[0163] As a preferred example, the frequency domain decomposition module 903 can filter the second tensor in eight directions through an eight-directional filter to obtain a feature image in each direction. Each feature map only represents information in one direction, so that the high frequency band of the first image can be observed from eight directions, and the differences in different directions can be obtained from various angles. This is conducive to fine-grained classification and recognition during the training process.

[0164] The frequency domain decomposition module 903 may also perform Fourier transform on the feature image to obtain a plurality of frequency domain transform images containing high frequency component distribution information.

[0165] In the feature images in all directions, the high-frequency component area obtains texture or contour information. After the feature image is subjected to Fourier (FT) transform, the high-frequency components are concentrated in the central area, and the low-frequency components are scattered in other areas of the image. Subsequent convolution processing can be regarded as discarding the statistical processing of the low frequency and retaining and balancing the statistical part of the high frequency.

[0166] When a traditional convolutional network is trained, its filter operator only extracts the contour and cannot extract the distribution of the image. Therefore, compared with traditional technology, in the embodiment of the present disclosure, after the FT transformation, the high-frequency components are concentrated and the low-frequency components are dispersed, and the image is distributed twice. This is then connected to the convolution pooling process, which can partially solve the problem of the high-dimensional unexplainable deep convolution layer (that is, the accuracy of deep learning is unpredictable). Compared with traditional technology, the device of the present disclosure does not need to strictly distribute the image uniformly according to the training requirements, and can also better perform classification and recognition. The device of the present disclosure adds the extraction of image statistical information to the traditional extraction of image semantic information. This multi-responsibility mode can better perform recognition, especially the cluster classification and recognition of discrete objects, and solve the problem of fine-grained classification of discrete objects that cannot be solved by traditional frequency domain decomposition, machine learning, or convolutional neural network alone.

[0167] Therefore, the frequency domain decomposition module 903 can also perform convolution pooling processing on multiple frequency domain transformed images and output a frequency domain tensor.

[0168] In this embodiment, the obtained multiple frequency domain transformation images are convoluted and pooled three times, and the size of the frequency domain transformation images is downsampled to the output dimension of the capsule network. A 3*3 convolution kernel can be used, and the padding attribute (padding) is 1 to keep the input and output dimensions consistent. The pooling uses a 2*2 kernel, and each layer reduces the size of the feature image by half. In this way, a frequency domain tensor with the same dimension as the capsule tensor can be obtained for subsequent fusion processing of tensor data.

[0169] The frequency domain decomposition module 903 may also decompose the frequency domain tensor through one-dimensional convolution.

[0170] Since the more convolution pooling layers there are, the more data volume is obtained. After the frequency domain decomposition module processes the sample data, the actual amount of tensor data obtained is also large. For this reason, this embodiment uses one-dimensional convolution to directly fuse the frequency domain tensor after convolution pooling processing, which can reduce the data volume, reduce training parameters, and speed up training in the subsequent cross-layer connection process.

[0171] The fusion module 904 can be used to implement Figure 1 Step S400 of the discrete object classification method shown in FIG. 1 is used to fuse the capsule tensor and the frequency domain tensor to obtain tensor fusion data of the first image. Specifically, the fusion module mainly fuses the tensor data output by the capsule network module and the frequency domain decomposition module to achieve Figure 6 Steps S401 to S402 of the discrete object classification method shown in FIG.

[0172] Specifically, the fusion module 904 performs a fully connected feature combination of the capsule tensor and the frequency domain tensor through a one-dimensional convolution method to obtain a feature fusion tensor. The fusion module uses a one-dimensional convolution method through a fully connected layer to achieve an equivalent mapping of the capsule tensor and the frequency domain tensor to the fully connected output result. Different from traditional neural networks, the embodiment of the present disclosure abandons the softmax layer of the neural network, uses one-dimensional convolution in the fully connected layer, and uses matrix acceleration operations to speed up training and has stronger scalability.

[0173] The fusion module 904 also reduces the dimension of the feature fusion tensor by one-dimensional convolution according to the number of preset categories to obtain tensor fusion data with the same number of preset categories.

[0174] The capsule tensor and the frequency domain tensor are combined by one-dimensional convolution to perform full-connection feature combination, and then the dimension is reduced to the same number as the preset number of categories by one-dimensional convolution. For example, in the example given in this embodiment, the finished tea is classified into 18 categories, and the tensor fusion data obtained after one-dimensional convolution processing is 18*1.

[0175] The classification module 905 can be used to implement Figure 1 Step S500 of the discrete object classification method shown in the figure is used to determine the category of the discrete object corresponding to the first image from the preset categories according to the tensor fusion data. The tensor fusion data is a combination of the basic features and frequency domain features of the first image in different dimensions. In the process of training the recognition model, the classification module 905 of this embodiment continuously trains the weights of the tensor fusion data through a conventional loss function until the corresponding category is correct or infinitely close to the correct one, and the training of the recognition model is completed; then it can be directly used in the classification and recognition of discrete objects.

[0176] For the trained recognition model, in the classification and recognition process, the tensor fusion data already has a fixed weight and can be mapped to the preset category. Therefore, the classification module does not need to perform loss function correction on the tensor fusion data, and can directly map to the preset category to complete the recognition process.

[0177] In some embodiments, the apparatus further includes: a recognition model 900, which can be used to implement Figure 2 The step S600 of the classification method of discrete objects shown in FIG. 1 is used to classify and identify discrete objects; the recognition model can use an open source deep learning tool (such as PyTorch or TensorFlow) to build a network, which has a first acquisition module 901, a capsule network module 902, a frequency domain decomposition module 903, a fusion module 904 and a classification module 905, which can be based on Figure 2 Steps S100 to S500 of the discrete object classification method shown are trained to have the function of discrete object classification.

[0178] Furthermore, this embodiment may also include a second acquisition module 906 for implementing Figure 8 Step S601 of the discrete object classification method shown in the figure is to collect a second image of the discrete object; the second image is the image to be recognized, and the user takes a photo of the discrete object and uploads the photo to the recognition model, which decodes and compresses the photo and then inputs it into the recognition model to execute Figure 8 Step S602 of the discrete object classification method shown is to perform Figure 2 The identification process of steps S100 to S500 of the discrete object classification method is shown.

[0179] Output module 907, used to implement Figure 8 In step S603 of the discrete object classification method shown, after recognizing the second image through the recognition model, the category information of the discrete object corresponding to the second image is output. The recognition accuracy probability of each category can also be output and arranged from high to low to verify or analyze the recognition effect of the recognition model.

[0180] Fig.10 A schematic diagram of the hardware structure of a discrete object classification device provided in an embodiment of the present disclosure is shown.

[0181] The discrete object classification device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0182] Specifically, the processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0183] The memory 1002 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In one example, the memory 1002 may include a removable or non-removable (or fixed) medium, or the memory 1002 is a non-volatile solid-state memory. The memory 1002 may be inside or outside the integrated gateway disaster recovery device.

[0184] The memory 1002 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0185] The processor 1001 reads and executes the computer program instructions stored in the memory 1002 to implement Figure 1 The method / steps S100 to S600 in the embodiment shown in the figure are as follows: Figure 1 The corresponding technical effects achieved by executing the methods / steps in the example shown are not repeated here for the sake of brevity.

[0186] In one example, the discrete object classification device may further include a communication interface 1003 and a bus 1010. Fig.10 As shown, the processor 1001, the memory 1002, and the communication interface 1003 are connected via a bus 1010 and communicate with each other.

[0187] The communication interface 1003 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present disclosure.

[0188] Bus 1010 includes hardware, software or both, and the components of online data flow billing equipment are coupled to each other. For example, but not limitation, the bus may include Accelerated Graphics Port (AGP) or other graphics bus, Enhanced Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), Hyper Transport (HT) interconnection, Industry Standard Architecture (ISA) bus, InfiniBand interconnection, Low Pin Count (LPC) bus, Memory bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1010 may include one or more buses. Although the present disclosure embodiment describes and shows a specific bus, the present disclosure considers any suitable bus or interconnection.

[0189] In addition, in combination with the discrete object classification method in the above embodiment, the embodiment of the present disclosure may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any discrete object classification method in the above embodiment is implemented.

[0190] It should be clear that the present disclosure is not limited to the specific configurations and processes described above and shown in the figures. For the sake of simplicity, a detailed description of the known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present disclosure is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present disclosure.

[0191] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present disclosure are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0192] It should also be noted that the exemplary embodiments mentioned in the present disclosure describe some methods or systems based on a series of steps or devices. However, the present disclosure is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0193] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0194] The above is only a specific implementation of the present disclosure. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present disclosure is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure.

Claims

1. A method for classifying discrete objects, characterized in that: The method comprises: Acquiring a first image of the discrete object; Using a capsule network, extracting target features from the first image, and outputting a capsule tensor of the first image; The capsule tensor is fused with the frequency domain tensor of the first image to obtain tensor fusion data of the first image; wherein the frequency domain tensor is obtained by extracting frequency domain information features of the first image by frequency domain decomposition; According to the tensor fusion data, a category of the discrete object corresponding to the first image is determined from preset categories.

2. The method for classifying discrete objects according to claim 1, characterized in that: The capsule network includes a convolutional layer, a plurality of main capsule layers, and a digital capsule layer; The step of using a capsule network to extract target features from the first image and outputting a capsule tensor of the first image includes: Inputting the first image into the convolution layer for convolution calculation, and the convolution layer outputs target feature data of the first image; Input the target feature data into the plurality of main capsule layers accordingly, combine the target feature data through the plurality of main capsule layers, and output a first tensor; The first tensor is input into the digital capsule layer, the digital capsule layer has a plurality of digital capsules, and each of the preset categories corresponds to a digital capsule; the plurality of digital capsules all use the first tensor as input data, and are calculated through a dynamic routing algorithm to output the capsule tensor.

3. The method for classifying discrete objects according to claim 1, characterized in that: The extracting frequency domain information features of the first image by frequency domain decomposition includes: Performing frequency domain decomposition on the digital signal of the first image to obtain a second tensor; wherein the second tensor is data of high-frequency components decomposed from the first image, or the second tensor is data of low-frequency components mixed with high-frequency components decomposed from the first image; Recursively decomposing the low-frequency component toward the high-frequency component; Performing multiple direction filtering on the second tensor to obtain feature images in various directions; Performing Fourier transformation on the characteristic image to obtain a plurality of frequency domain transformation images containing the high frequency component distribution information; Perform convolution pooling processing on the multiple frequency domain transformed images, and output the frequency domain tensor; wherein the frequency domain tensor has the same dimension as the capsule tensor.

4. The method for classifying discrete objects according to claim 3, characterized in that: The step of performing convolution pooling processing on the multiple frequency domain transformed images and outputting the frequency domain tensor comprises: The frequency domain transform image is subjected to multiple convolution pooling processes, and the frequency domain transform image is downsampled to the same output dimension as the capsule network to obtain the frequency domain tensor with the same dimension as the capsule tensor, and the frequency domain tensor is output.

5. The method for classifying discrete objects according to claim 1, characterized in that: Fusing the capsule tensor and the frequency domain tensor to obtain tensor fusion data of the first image includes: By one-dimensional convolution, the capsule tensor and the frequency domain tensor are fully connected to obtain a feature fusion tensor; The feature fusion tensor is reduced in dimension by a one-dimensional convolution according to the number of the preset categories to obtain the tensor fusion data with the same number as the preset categories.

6. The method for classifying discrete objects according to any one of claims 1 to 5, characterized in that: After determining the category of the discrete object corresponding to the first image from preset categories according to the tensor fusion data, the method further includes: A preset recognition model is trained by the classification method of the discrete objects to classify and recognize the discrete objects; Acquiring a second image of the discrete object; recognizing the second image by using the recognition model; Output the category of the discrete object corresponding to the second image.

7. A discrete object classification device, characterized in that: The device comprises: A first acquisition module, used for acquiring a first image of the discrete object; A capsule network module, configured to extract target features from the first image using a capsule network, and output a capsule tensor of the first image; A frequency domain decomposition module, configured to extract frequency domain information features of the first image by frequency domain decomposition, and output a frequency domain tensor of the first image; A fusion module: used for fusing the capsule tensor and the frequency domain tensor to obtain tensor fusion data of the first image; Classification module: used to determine the category of the discrete object corresponding to the first image from preset categories based on the tensor fusion data.

8. The discrete object classification device according to claim 7, characterized in that: The device also includes: A recognition model, used for classifying and identifying the discrete objects; the recognition model is obtained by training the first acquisition module, the capsule network module, the frequency domain decomposition module, the fusion module and the class module; A second acquisition module, used for acquiring a second image of the discrete object; The output module is used to output the category information of the discrete object corresponding to the second image after the second image is recognized by the recognition model.

9. A discrete object classification device, characterized in that: The device comprises: A processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the discrete object classification method according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the discrete object classification method according to any one of claims 1 to 6 is implemented.