Image recognition device and method
By extracting and processing features and texture features from the image recognition device, and combining sliding window and average pooling techniques, the problem of low-contrast image recognition difficulty is solved, and efficient and accurate image object recognition is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INSTITUTE FOR INFORMATION INDUSTRY
- Filing Date
- 2024-12-03
- Publication Date
- 2026-05-29
AI Technical Summary
Existing image recognition technologies struggle to correctly identify objects in low-contrast images, especially considering the rotational nature of texture features, and edge computing devices have limited computing power.
The image recognition device performs feature extraction operations to generate a feature map, and uses a sliding window to calculate the variation feature values of adjacent pixels to generate a texture feature map. Combined with the average pooling feature map, the ability to identify texture features is improved. At the same time, it determines whether the image is a low-contrast image and extracts texture features only when necessary.
It improves the accuracy and efficiency of image object recognition, simplifies the computational burden of texture feature extraction, and significantly enhances recognition performance, especially when processing low-contrast images.
Smart Images
Figure CN122116090A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to an image recognition device and method, and more particularly to an image recognition device and method for enhancing the recognition of low-contrast images. Background Technology
[0002] As machine learning applications become increasingly diverse, image recognition models are frequently used on edge computing devices. However, edge computing devices are limited by hardware specifications, resulting in limited computing power.
[0003] Furthermore, existing image recognition technologies rely on global and local pixel statistics to extract texture features from images. However, the extraction results are not significant enough, and it is difficult to discern the rotational nature of texture features within the image. As a result, existing technologies often fail to correctly identify objects in low-contrast images.
[0004] Therefore, enhancing the ability to extract texture features in image recognition is a goal that the industry urgently needs to strive for. Summary of the Invention
[0005] To address the aforementioned problems, this disclosure proposes an image recognition device comprising a storage device and a processor. The storage device is used to store an image. The processor is electrically connected to the storage device and is used to perform the following operations: performing a feature extraction operation based on the image to generate a feature map; performing a texture feature extraction operation based on the feature map to generate a texture feature map; and performing a classification operation based on the texture feature map to identify an object in the image.
[0006] In one embodiment of the present invention, the feature extraction operation includes the following operations: using a sliding window, calculating multiple variant feature values of multiple groups of adjacent pixels in the feature map; and generating the texture feature map based on the multiple variant feature values.
[0007] In one embodiment of the present invention, the operation of calculating the plurality of variant feature values further includes: using the sliding window to calculate a plurality of average values and a plurality of variants of the plurality of adjacent pixels; and calculating the plurality of variant feature values based on the plurality of average values and the plurality of variants.
[0008] In one embodiment of the invention, the feature extraction operation includes the following operations: performing an average pooling operation based on the feature map to generate an average pooled feature map; and generating the texture feature map based on the plurality of variable feature values and the average pooled feature map.
[0009] In one embodiment of the present invention, the processor is further configured to perform the following operations: before performing the feature extraction operation, determine whether the image is a low-contrast image; and in response that the image is not the low-contrast image, not perform the texture feature extraction operation and perform the classification operation based on the feature map to identify the object in the image.
[0010] In one embodiment of the present invention, the operation of determining whether the image is the low-contrast image further includes: calculating a plurality of brightness values of a plurality of pixel values of the image; calculating a dispersion index of the plurality of brightness values; and determining that the image is the low-contrast image in response to the dispersion index being lower than a dispersion threshold.
[0011] This disclosure also provides an image recognition method applicable to an electronic device, the steps of which include: performing a feature extraction operation based on an image to generate a feature map; performing a texture feature extraction operation based on the feature map to generate a texture feature map; and performing a classification operation based on the texture feature map to identify an object in the image.
[0012] In one embodiment of the present invention, the feature extraction operation includes the following steps: using a sliding window, calculating multiple variant feature values of multiple groups of adjacent pixels in the feature map; and generating the texture feature map based on the multiple variant feature values.
[0013] In one embodiment of the present invention, the step of calculating the plurality of variation feature values further includes: using the sliding window to calculate a plurality of average values and a plurality of variation numbers of the plurality of adjacent pixels; and calculating the plurality of variation feature values based on the plurality of average values and the plurality of variation numbers.
[0014] In one embodiment of the present invention, the feature extraction operation includes the following steps: performing an average pooling operation based on the feature map to generate an average pooling feature map; and generating the texture feature map based on the plurality of variant feature values and the average pooling feature map.
[0015] In one embodiment of the present invention, the method further includes: determining whether the image is a low-contrast image before performing the feature extraction operation; and in response that the image is not the low-contrast image, not performing the texture feature extraction operation and performing the classification operation based on the feature map to identify the object in the image.
[0016] In one embodiment of the present invention, the step of determining whether the image is the low-contrast image further includes: calculating a plurality of brightness values of a plurality of pixel values of the image; calculating a dispersion index of the plurality of brightness values; and determining that the image is the low-contrast image in response to the dispersion index being lower than a dispersion threshold.
[0017] It should be understood that the foregoing general description and the following specific description are merely exemplary and explanatory, and are intended to provide further explanation of the claimed disclosure. Attached Figure Description
[0018] To make the above and other objects, features, advantages and embodiments disclosed herein more apparent and understandable, the accompanying drawings are described below:
[0019] Figure 1 This is a schematic diagram of the image recognition device in the first embodiment of this disclosure;
[0020] Figure 2 This is a schematic diagram illustrating the operation of the image recognition device generating recognition results based on images in some embodiments of this disclosure;
[0021] Figure 3 This is a schematic diagram illustrating the operation of texture feature extraction in some embodiments of this disclosure; and
[0022] Figure 4 This is a flowchart illustrating the image recognition method in the second embodiment of this disclosure.
[0023] [Symbol Explanation]
[0024] 1: Image recognition device
[0025] 12: Processor
[0026] 14: Storage
[0027] IMG: Images
[0028] FM: Feature Map
[0029] TFM: Texture Feature Map
[0030] RS: Identification Results
[0031] OP2, OP4, OP6, OP4_2, OP4_4, OP4_6: Operation
[0032] VFM: Mutation Feature Map
[0033] APFM: Average Pooling Feature Map
[0034] 200: Image Recognition Methods
[0035] S202, S204, S206: Steps Detailed Implementation
[0036] To make the description of this disclosure more detailed and complete, reference can be made to the accompanying drawings and the various embodiments described below, in which the same numbers represent the same or similar elements.
[0037] Please refer to Figure 1 This is a schematic diagram of the image recognition device 1 in the first embodiment of this disclosure. The image recognition device 1 includes a processor 12 and a storage device 14, wherein the processor 12 is electrically connected to the storage device 14. The image recognition device 1 is used to enhance the texture features in an image in order to recognize objects in the image.
[0038] In some embodiments, processor 12 may include a central processing unit (CPU), a graphics processing unit (GPU), a multiprocessor, a distributed processing system, an application-specific integrated circuit (ASIC), and / or a suitable computing unit.
[0039] Storage 14 is used to store an image. In some embodiments, storage 14 may include semiconductor or solid-state memory, magnetic tape, portable computer disk, random access memory (RAM), read-only memory (ROM), hard disk and / or optical disk.
[0040] It should be noted that the image can be an optical image containing multiple color value pixels, or a thermal image containing multiple temperature pixels.
[0041] First, in order to extract the features of the image, the image recognition device 1 performs feature extraction operations on the image and generates a feature map.
[0042] Next, in order to enhance the extraction of texture features from the image, the image recognition device 1 performs texture feature extraction operations on the feature map and generates a texture feature map.
[0043] Finally, the image recognition device 1 performs classification operations on the texture feature map to identify objects in the image.
[0044] Specifically, the processor 12 performs a feature extraction operation based on the image to generate a feature map; the processor 12 performs a texture feature extraction operation based on the feature map to generate a texture feature map; and the processor 12 performs a classification operation based on the texture feature map to identify an object in the image.
[0045] For details on the operation of image recognition device 1, please refer to [link / reference]. Figure 2 This is a schematic diagram of the operation of the image recognition device 1 generating recognition result RS based on the image IMG in some embodiments of this disclosure.
[0046] First, during operation OP2, the image recognition device 1 performs feature extraction calculations on the image IMG and generates a feature map FM.
[0047] In some embodiments, taking a convolutional neural network (CNN) as an example, the image recognition device 1 uses multiple convolutional layers to perform feature extraction operations on the image IMG and generate a feature map FM.
[0048] Specifically, the feature extraction operation includes the following steps: performing multiple convolutional layer operations based on the image to generate the feature map.
[0049] Next, during operation OP4, the image recognition device 1 performs texture feature extraction operations on the feature map FM and generates a texture feature map TFM.
[0050] In some embodiments, the image recognition device 1 uses a sliding window to calculate the variation feature values of multiple adjacent pixels in the feature map FM.
[0051] Specifically, the feature extraction operation includes the following operations: using a sliding window, calculating multiple variant feature values of multiple groups of adjacent pixels in the feature map; and generating the texture feature map based on the multiple variant feature values.
[0052] For example, processor 12 uses a sliding window to select a specific number of pixels (e.g., 1*2, 2*1, 2*2, or 3*2 pixels) in the feature map FM to form a group of adjacent pixels, and calculates the variant feature values of the adjacent pixels. Then, it selects the next group of adjacent pixels by translation (e.g., translating one pixel to the right) and calculates the corresponding variant feature values. By repeating this translation and calculation process, processor 12 can calculate the variant feature values of multiple groups of adjacent pixels and generate the texture feature map TFM accordingly.
[0053] In some embodiments, when calculating the variation feature value, the image recognition device 1 calculates the average value and the number of variations of each group of adjacent pixels, and calculates the corresponding variation feature value accordingly.
[0054] Specifically, the operation of calculating the plurality of variant feature values further includes: using the sliding window to calculate a plurality of average values and a plurality of variants of the plurality of adjacent pixels; and calculating the plurality of variant feature values based on the plurality of average values and the plurality of variants.
[0055] For example, processor 12 calculates the variant eigenvalues based on the following equation.
[0056] [Formula 1]
[0057]
[0058] Where FV is the variation feature value corresponding to a set of adjacent pixels, σ 2 denoted as σ, where μ is the variation of the values of the adjacent pixels in the group, and μ is the average value of the values of the adjacent pixels in the group.
[0059] In some embodiments, if only the variability feature values are used as pixels in the texture feature map (TFM), excessive deviation in local spatial features may affect the overall texture feature representation. Therefore, the image recognition device 1 also calculates an average pooling feature map and combines the average pooling feature map with the variability feature map composed of variability feature values to generate the texture feature map (TFM). In this way, the image recognition device 1 can avoid overfitting of the texture feature map (TFM) and reduce its excessive feature deviation.
[0060] Specifically, the feature extraction operation includes the following operations: performing an average pooling operation based on the feature map to generate an average pooled feature map; and generating the texture feature map based on the plurality of variable feature values and the average pooled feature map.
[0061] For operational details regarding the variability feature map and the average pooling feature map, please refer to [link / reference]. Figure 3 It illustrates the details of the texture feature extraction operation in OP4 in some embodiments.
[0062] like Figure 3 As shown, the image recognition device 1 calculates the variation feature values and generates a variation feature map (VFM) during operation OP4_2. On the other hand, the image recognition device 1 performs an average pooling operation during operation OP4_4 and generates an average pooling feature map (APFM).
[0063] After calculating the variation feature map (VFM) and the average pooling feature map (APFM), in operation OP4_6, the image recognition device 1 combines the variation feature map (VFM) and the average pooling feature map (APFM) to generate the texture feature map (TFM).
[0064] In some embodiments, the image recognition device 1 calculates the external product of the variation feature map (VFM) and the average pooling feature map (APFM) to combine the two and generate a texture feature map (TFM).
[0065] Please return Figure 2 After generating the texture feature map (TFM), during operation OP6, the image recognition device 1 performs classification operations on the texture feature map (TFM) and generates the recognition result (RS) to identify objects in the image.
[0066] In some embodiments, taking a convolutional neural network as an example, the image recognition device 1 uses a fully connected layer to perform classification operations on the texture feature map (TFM) and generate an recognition result (RS), wherein the recognition result (RS) includes the position of at least one object in the image (IMG) and the attributes of the object (e.g., vehicle, human body, animal).
[0067] Specifically, the classification operation includes the following operation: performing a fully connected layer operation based on the texture feature map to produce an identification result, wherein the identification result is used to indicate a position of the object in the image and an object attribute of the object.
[0068] In some embodiments, before performing image recognition, the image recognition device 1 first determines whether the image IMG is a low-contrast image. If the image IMG is a low-contrast image, image recognition is performed in the manner described above; otherwise, if the image IMG is not a low-contrast image, texture feature extraction is skipped (e.g., operation OP4), and classification operation is performed directly on the feature map FM after feature extraction (e.g., operation OP2) to generate the recognition result RS.
[0069] It should be noted that the pixel values in low-contrast images are relatively concentrated, making it more difficult to identify contours, shapes, and textures in the images during image recognition. Therefore, the image recognition device 1 enhances the extraction of texture features for low-contrast images.
[0070] Specifically, the processor 12 is further configured to perform the following operations: before performing the feature extraction operation, determine whether the image is a low-contrast image; and in response that the image is not the low-contrast image, not perform the texture feature extraction operation and perform the classification operation based on the feature map to identify the object in the image.
[0071] In this way, the image recognition device 1 can trigger texture feature extraction calculations only when low-contrast images need to be identified, thereby improving texture feature extraction capabilities and increasing recognition accuracy. In other cases, the image recognition device 1 performs image recognition in a relatively fast manner, improving recognition efficiency.
[0072] Image recognition device 1 can determine whether an image IMG is a low-contrast image in different ways. In some embodiments, since the brightness histogram values of low-contrast images often concentrate in the middle range of brightness, with lower values at both high and low brightness levels on the sides, image recognition device 1 determines whether an image IMG is a low-contrast image by calculating the dispersion of the image IMG brightness histogram.
[0073] Specifically, the operation of determining whether an image is a low-contrast image further includes: calculating multiple brightness values of multiple pixel values of the image; calculating a dispersion index of the multiple brightness values; and determining that the image is a low-contrast image in response to the dispersion index being lower than a dispersion threshold.
[0074] For example, processor 12 calculates the standard deviation of the brightness values as a dispersion index. If the standard deviation is lower than the dispersion threshold, it indicates that the brightness values are concentrated and the image IMG is a low-contrast image.
[0075] In some embodiments, the image recognition device 1 determines whether the image IMG is a low-contrast image by calculating the relationship between the maximum and minimum values of the image IMG brightness value.
[0076] Specifically, the operation of determining whether an image is a low-contrast image further includes: calculating multiple brightness values of multiple pixel values of the image; calculating a contrast index based on a maximum value and a minimum value among the multiple brightness values; and determining that the image is a low-contrast image in response to the contrast index being lower than a contrast threshold.
[0077] For example, processor 12 uses the Michelson comparison formula (as shown in Equation 2 below) to calculate the comparison index.
[0078] [Formula 2]
[0079] I M =(I max -I min ) / (I max +I min )
[0080] Where I M For comparison metrics, I max I represents the maximum brightness. min This represents the minimum brightness value. The lower the contrast of the image IMG, the lower the value of the contrast index. Therefore, when the contrast index is lower than the contrast threshold, the image recognition device 1 determines that the image IMG is a low-contrast image.
[0081] In some embodiments, the image recognition device 1 calculates the image gradient of the image IMG to determine the degree of brightness change of the pixels in the image IMG, and thereby determines whether the image IMG is a low-contrast image.
[0082] Specifically, the operation of determining whether an image is a low-contrast image further includes: calculating an image gradient of the image using a filter; calculating a gradient index corresponding to the image gradient; and determining that the image is a low-contrast image in response to the gradient index being lower than a gradient threshold.
[0083] For example, since the gradient mean and variance of low-contrast images are relatively low, the image recognition device 1 first uses a Sobel or Laplacian filter to calculate the image gradient of the image IMG, and then performs statistical analysis on the image gradient (e.g., calculate the gradient mean or gradient standard deviation). If the calculated value is lower than the gradient threshold, it indicates that the image IMG is a low-contrast image.
[0084] It should be noted that the image recognition device 1 can determine whether an image IMG is a low-contrast image by using one or a combination of the multiple methods in the above embodiments, and this disclosure is not limited to this.
[0085] In summary, the image recognition device 1 disclosed herein improves the accuracy of object recognition in images by enhancing the extraction of texture features, and the calculation method for texture feature extraction is simple without adding excessive computational burden. Furthermore, by first determining whether the image is a low-contrast image, the image recognition device 1 can determine whether texture feature extraction is necessary, thereby achieving a more efficient image recognition technology.
[0086] Please refer to Figure 4 This is a flowchart of the image recognition method 200 in the second embodiment of this disclosure. The image recognition method 200 includes steps S202 to S206. The image recognition method 200 is used to enhance the texture features in an image in order to identify objects in the image. The image recognition method 200 can be executed by an electronic device (e.g., the image recognition device 1 in the first embodiment).
[0087] First, in step S202, the electronic device performs a feature extraction operation based on an image to generate a feature map.
[0088] Next, in step S204, the electronic device performs a texture feature extraction operation based on the feature map to generate a texture feature map.
[0089] Finally, in step S206, the electronic device performs a classification operation based on the texture feature map to identify an object in the image.
[0090] In some embodiments, the feature extraction operation includes the following steps: the electronic device performs multiple convolutional layer operations based on the image to generate the feature map.
[0091] In some embodiments, the feature extraction operation includes the following steps: the electronic device uses a sliding window to calculate multiple variant feature values of multiple groups of adjacent pixels in the feature map; and the electronic device generates the texture feature map based on the multiple variant feature values.
[0092] In some embodiments, the step of calculating the plurality of variation feature values further includes: the electronic device using the sliding window to calculate a plurality of average values and a plurality of variation numbers of the plurality of adjacent pixels; and the electronic device calculating the plurality of variation feature values based on the plurality of average values and the plurality of variation numbers.
[0093] In some embodiments, the feature extraction operation includes the following steps: the electronic device performs an average pooling operation based on the feature map to generate an average pooling feature map; and the electronic device generates the texture feature map based on the plurality of variant feature values and the average pooling feature map.
[0094] In some embodiments, the classification operation includes the following steps: the electronic device performs a fully connected layer operation based on the texture feature map to generate an identification result, wherein the identification result is used to indicate a position of the object in the image and an object attribute of the object.
[0095] In some embodiments, the image recognition method 200 further includes determining whether the image is a low-contrast image before performing the feature extraction operation; and in response that the image is not the low-contrast image, not performing the texture feature extraction operation and performing the classification operation based on the feature map to identify the object in the image.
[0096] In some embodiments, the step of determining whether the image is the low-contrast image further includes: the electronic device calculating a plurality of brightness values of a plurality of pixel values of the image; the electronic device calculating a dispersion index of the plurality of brightness values; and in response to the dispersion index being lower than a dispersion threshold, the electronic device determining that the image is the low-contrast image.
[0097] In some embodiments, the step of determining whether the image is the low-contrast image further includes: the electronic device calculating a plurality of brightness values of a plurality of pixel values of the image; the electronic device calculating a contrast index based on a maximum value and a minimum value among the plurality of brightness values; and in response to the contrast index being lower than a contrast threshold, the electronic device determining that the image is the low-contrast image.
[0098] In some embodiments, the step of determining whether the image is the low-contrast image further includes: the electronic device calculating an image gradient of the image using a filter; the electronic device calculating a gradient index corresponding to the image gradient; and in response to the gradient index being lower than a gradient threshold, the electronic device determining that the image is the low-contrast image.
[0099] In summary, the image recognition method 200 proposed in this disclosure improves the accuracy of object recognition in images by enhancing the extraction of texture features. Furthermore, the calculation method for texture feature extraction is simple and does not add excessive computational burden. In addition, by first determining whether the image is a low-contrast image, the image recognition method 200 can determine whether texture feature extraction is necessary, thereby achieving a more efficient image recognition technology.
[0100] Although several embodiments have been described in detail above as examples, the image recognition apparatus and method disclosed herein can also be implemented in other systems, hardware, software, storage media, or combinations thereof. Therefore, the scope of protection of this disclosure should not be limited to the specific implementations described in the embodiments disclosed herein, but should be determined by the appended claims.
[0101] It will be apparent to those skilled in the art to which this disclosure pertains that various modifications and variations can be made to the structure of this disclosure without departing from its scope or spirit. In view of the foregoing, the scope of protection of this disclosure also covers modifications and variations made within the scope of the appended claims.
Claims
1. An image recognition device, characterized in that, Include: A storage device for storing an image; and A processor, electrically connected to the memory, is used to perform the following operations: A feature extraction operation is performed on the image to generate a feature map; Based on this feature map, a texture feature extraction operation is performed to generate a texture feature map; and A classification operation is performed based on the texture feature map to identify an object in the image.
2. The image recognition device as described in claim 1, characterized in that, The feature extraction operation includes the following steps: Using a sliding window, calculate multiple variation feature values of multiple groups of adjacent pixels in the feature map; and The texture feature map is generated based on the multiple variant feature values.
3. The image recognition device as described in claim 2, characterized in that, The operation of calculating the multiple variant feature values further includes: Using this sliding window, calculate multiple average values and multiple variances of the multiple groups of adjacent pixels; and Based on the multiple average values and the multiple variances, the multiple variance characteristic values are calculated.
4. The image recognition device as described in claim 2, characterized in that, The feature extraction operation includes the following steps: An average pooling operation is performed on the feature map to produce an average pooling feature map; and The texture feature map is generated based on the multiple variant feature values and the average pooling feature map.
5. The image recognition device as described in claim 1, characterized in that, The processor is further used to perform the following operations: Before performing this feature extraction operation, it is determined whether the image is a low-contrast image; and In response to the image not being the low-contrast image, the texture feature extraction operation is not performed and the classification operation is performed based on the feature map to identify the object in the image.
6. The image recognition device as described in claim 5, characterized in that, The process of determining whether an image is a low-contrast image further includes: Calculate multiple brightness values for multiple pixel values in the image; Calculate a dispersion index for the plurality of brightness values; and If the dispersion index is below a dispersion threshold, the image is determined to be a low-contrast image.
7. An image recognition method, characterized in that, Applicable to an electronic device, the steps include: A feature extraction operation is performed on an image to generate a feature map; Based on this feature map, a texture feature extraction operation is performed to generate a texture feature map; and A classification operation is performed based on the texture feature map to identify an object in the image.
8. The image recognition method as described in claim 7, characterized in that, The feature extraction operation includes the following steps: Using a sliding window, calculate multiple variation feature values of multiple groups of adjacent pixels in the feature map; and The texture feature map is generated based on the multiple variant feature values.
9. The image recognition method as described in claim 8, characterized in that, The step of calculating the plurality of variant feature values further includes: Using this sliding window, calculate multiple average values and multiple variances of the multiple groups of adjacent pixels; and Based on the multiple average values and the multiple variances, the multiple variance characteristic values are calculated.
10. The image recognition method as described in claim 8, characterized in that, The feature extraction operation includes the following steps: An average pooling operation is performed on the feature map to produce an average pooling feature map; and The texture feature map is generated based on the multiple variant feature values and the average pooling feature map.
11. The image recognition method as described in claim 7, characterized in that, Further includes: Before performing this feature extraction operation, it is determined whether the image is a low-contrast image; and In response to the image not being the low-contrast image, the texture feature extraction operation is not performed and the classification operation is performed based on the feature map to identify the object in the image.
12. The image recognition method as described in claim 11, characterized in that, The step of determining whether the image is a low-contrast image further includes: Calculate multiple brightness values for multiple pixel values in the image; Calculate a dispersion index for the plurality of brightness values; and If the dispersion index is below a dispersion threshold, the image is determined to be a low-contrast image.