Image recognition method, device, electronic device and storage medium
The image recognition method improves accuracy by employing color space recognition, inter-class variance segmentation, and motion estimation to enhance target contour detection, addressing the challenges of scenery and lighting in video images.
Patent Information
- Application Number
- CN202210573459.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-24
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-05-24
AI Technical Summary
Image recognition accuracy is low due to the influence of factors such as scenery and lighting in video images, leading to the inability to recognize characters or specific objects, which affects the user's viewing experience.
An image recognition method involving color space recognition, image positioning, inter-class variance segmentation, edge detection, and motion estimation to improve accuracy by reducing the influence of background pixels and enhancing target contour detection.
The method enhances image recognition accuracy by accurately segmenting foreground objects from background, improving the precision of target identification and expanding its applicability across various scenarios.
Smart Images

Figure CN114913463B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to an image recognition method, device, electronic equipment and storage medium. Background Art
[0002] Currently, image recognition is widely used in all walks of life. For example, in the video-related field, it usually involves identifying people, specific objects, etc. in the video, and displaying relevant information about the identified people or objects to enhance the user's viewing experience. For example, the person appearing in the video is identified, and the profile of the identified person, the person's related video and other information are displayed.
[0003] However, due to the influence of other factors in the picture (such as scenery, lighting, etc.), image recognition may fail, that is, the image recognition accuracy is low. For example, in some video scenes, due to the excessive proportion of scenery in the video image or too little rendering of character information, it will lead to the inability to recognize the characters or specific objects appearing in the video image, resulting in users not having a good video viewing experience. Summary of the invention
[0004] The purpose of the embodiments of the present invention is to provide an image recognition method, device, electronic device and storage medium to improve the accuracy of image recognition. The specific technical solution is as follows:
[0005] In a first aspect of the present invention, an image recognition method is provided, comprising:
[0006] Get the image to be identified;
[0007] Performing color space recognition and image positioning on the image to be identified to obtain a first target area; wherein the first target area includes the target image to be identified;
[0008] Performing inter-class variance segmentation based on an adaptive threshold on the first target area to obtain a second target area;
[0009] Performing edge detection on the first target area to obtain a first target contour;
[0010] Based on the first target contour, performing motion estimation on the target image to obtain a second target contour;
[0011] Based on the second target contour, performing image segmentation on the second target area to obtain a target image to be identified;
[0012] Perform image recognition on the target image to be recognized to obtain a target image recognition result.
[0013] In an embodiment of the present invention, the step of performing color space recognition and image positioning on the image to be recognized to obtain the first target area includes:
[0014] Performing color space recognition on the image to be recognized to obtain multiple color regions;
[0015] Extracting features from each color region to obtain features of each color region;
[0016] Based on the features of the color regions, the color regions are merged to obtain a first target region.
[0017] In an embodiment of the present invention, before the step of merging the color regions based on the features of the color regions to obtain the first target region, the step further includes:
[0018] Performing color space recognition on the image to be recognized to determine the center point of each color area;
[0019] For each color region center point, calculate the distance from the center point to the center points of other color regions;
[0020] Based on the distances from the center points of each color area to the center points of other color areas, determining, among the center points of each color area, a maximum center point corresponding to the maximum distance and a minimum center point corresponding to the minimum distance;
[0021] Determine a target image candidate area based on the maximum center point and the minimum center point;
[0022] The step of merging the color regions based on the features of the color regions to obtain a first target region includes:
[0023] Based on the characteristics of the target image candidate region and the color regions, the color regions are merged to obtain a first target region.
[0024] In one embodiment of the present invention, the step of performing inter-class variance segmentation on the first target area based on an adaptive threshold to obtain a second target area includes:
[0025] Acquire the grayscale feature of the first target area;
[0026] Based on the grayscale feature of the first target area, obtaining the grayscale probability distribution of each pixel in the first target area;
[0027] Determining a membership function of the grayscale of each pixel in the first target area based on the grayscale probability distribution of each pixel in the first target area;
[0028] For each pixel in the first target area, based on the divided multiple granularity layer spaces, calculate the similarity between the pixel and its adjacent pixels;
[0029] For each pixel in the first target area, based on the membership function and the similarity between the pixel and its adjacent pixels, obtain the membership of the pixel to the foreground image;
[0030] Based on the membership degree of each pixel belonging to the foreground image, the first target region is segmented by inter-class variance to obtain a second target region.
[0031] In one embodiment of the present invention, the step of performing motion estimation on the target image based on the first target contour to obtain the second target contour includes:
[0032] For each target pixel point on the first target contour, calculating a first mean square error and a first minimum SAD value of the target pixel point and a first preset number of pixel points within a first preset range;
[0033] Determine the direction of the pixel point with the smallest first mean square error as the first minimum distortion direction;
[0034] Determine a pixel point within the first preset range and in the first minimum distortion direction as a second search starting point;
[0035] Calculating the second search starting point, and a second mean square error and a second minimum SAD value of a second preset number of pixel points within a second preset range;
[0036] Determine the direction of the pixel point with the smallest second mean square error as the second minimum distortion direction;
[0037] Determine a pixel point within the second preset range and in the second minimum distortion direction as a third search starting point;
[0038] Calculating the third search starting point and a third minimum SAD value among a third preset number of pixel points within a third preset range;
[0039] Determine a pixel point corresponding to a target minimum SAD value based on the first minimum SAD value, the second minimum SAD value, and the third minimum SAD value;
[0040] Calculate the target grayscale value corresponding to the target pixel based on the grayscale value of the pixel corresponding to the target minimum SAD value and the average grayscale value of the image to be identified;
[0041] A second target contour is obtained based on the target grayscale value of each target pixel point on the first target contour.
[0042] In an embodiment of the present invention, before the step of performing inter-class variance segmentation based on an adaptive threshold on the first target area to obtain the second target area, the step further includes:
[0043] Performing gradient adjustment on each pixel in the first target area to obtain an adjusted first target area;
[0044] The step of performing inter-class variance segmentation on the first target area based on an adaptive threshold to obtain a second target area includes:
[0045] The adjusted first target region is segmented based on inter-class variance of an adaptive threshold to obtain a second target region.
[0046] In one embodiment of the present invention, the method further comprises:
[0047] Based on the target image recognition result, the associated information preset for the recognition result is displayed.
[0048] In a second aspect of the present invention, there is also provided an image recognition device, comprising:
[0049] The module for obtaining the image to be identified is used to obtain the image to be identified;
[0050] A first target region acquisition module, used to perform color space recognition and image positioning on the image to be identified, to obtain a first target region; wherein the first target region includes the target image to be identified;
[0051] A second target region acquisition module, configured to perform inter-class variance segmentation on the first target region based on an adaptive threshold to obtain a second target region;
[0052] A first target contour acquisition module, used for performing edge detection on the first target area to acquire a first target contour;
[0053] A second target contour acquisition module, configured to perform motion estimation on the target image based on the first target contour to obtain a second target contour;
[0054] A module for acquiring an image of a target to be identified, configured to perform image segmentation on the second target area based on the second target contour to obtain an image of the target to be identified;
[0055] The image recognition module is used to perform image recognition on the target image to be recognized to obtain a target image recognition result.
[0056] In one embodiment of the present invention, the first target area acquisition module is used to:
[0057] Performing color space recognition on the image to be recognized to obtain multiple color regions;
[0058] Extracting features from each color region to obtain features of each color region;
[0059] Based on the features of the color regions, the color regions are merged to obtain a first target region.
[0060] In an embodiment of the present invention, before merging the color regions based on the features of the color regions to obtain the first target region, the method further includes:
[0061] Performing color space recognition on the image to be recognized to determine the center point of each color area;
[0062] For each color region center point, calculate the distance from the center point to the center points of other color regions;
[0063] Based on the distances from the center points of each color area to the center points of other color areas, determining, among the center points of each color area, a maximum center point corresponding to the maximum distance and a minimum center point corresponding to the minimum distance;
[0064] Determine a target image candidate area based on the maximum center point and the minimum center point;
[0065] The step of merging the color regions based on the features of the color regions to obtain a first target region includes:
[0066] Based on the characteristics of the target image candidate region and the color regions, the color regions are merged to obtain a first target region.
[0067] In one embodiment of the present invention, the second target area acquisition module is used to:
[0068] Acquire the grayscale feature of the first target area;
[0069] Based on the grayscale feature of the first target area, obtaining the grayscale probability distribution of each pixel in the first target area;
[0070] Determining a membership function of the grayscale of each pixel in the first target area based on the grayscale probability distribution of each pixel in the first target area;
[0071] For each pixel in the first target area, based on the divided multiple granularity layer spaces, calculate the similarity between the pixel and its adjacent pixels;
[0072] For each pixel in the first target area, based on the membership function and the similarity between the pixel and its adjacent pixels, obtain the membership of the pixel to the foreground image;
[0073] Based on the membership degree of each pixel belonging to the foreground image, the first target region is segmented by inter-class variance to obtain a second target region.
[0074] In one embodiment of the present invention, the second target profile acquisition module is used to:
[0075] For each target pixel point on the first target contour, calculating a first mean square error and a first minimum SAD value of the target pixel point and a first preset number of pixel points within a first preset range;
[0076] Determine the direction of the pixel point with the smallest first mean square error as the first minimum distortion direction;
[0077] Determine a pixel point within the first preset range and in the first minimum distortion direction as a second search starting point;
[0078] Calculating the second search starting point, and a second mean square error and a second minimum SAD value of a second preset number of pixel points within a second preset range;
[0079] Determine the direction of the pixel point with the smallest second mean square error as the second minimum distortion direction;
[0080] Determine a pixel point within the second preset range and in the second minimum distortion direction as a third search starting point;
[0081] Calculating the third search starting point and a third minimum SAD value among a third preset number of pixel points within a third preset range;
[0082] Determine a pixel point corresponding to a target minimum SAD value based on the first minimum SAD value, the second minimum SAD value, and the third minimum SAD value;
[0083] Calculate the target grayscale value corresponding to the target pixel based on the grayscale value of the pixel corresponding to the target minimum SAD value and the average grayscale value of the image to be identified;
[0084] A second target contour is obtained based on the target grayscale value of each target pixel point on the first target contour.
[0085] In one embodiment of the present invention, the device further comprises:
[0086] A gradient adjustment module, used for performing gradient adjustment on each pixel in the first target area to obtain an adjusted first target area;
[0087] The second target contour acquisition module is used to perform inter-class variance segmentation based on an adaptive threshold on the adjusted first target area to obtain a second target area.
[0088] In one embodiment of the present invention, it also includes:
[0089] The associated information display module is used to display the associated information preset for the recognition result based on the target image recognition result.
[0090] In another aspect of the present invention, an electronic device is provided, characterized in that it includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;
[0091] Memory, used to store computer programs;
[0092] The processor is used to implement any of the above-mentioned steps of the image recognition method when executing the program stored in the memory.
[0093] In another aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned image recognition methods is implemented.
[0094] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned image recognition methods.
[0095] The image recognition method provided by the embodiment of the present invention performs color space recognition and image positioning on the image to be recognized after obtaining the image to be recognized, obtains the first target area, and performs inter-class variance segmentation with an adaptive threshold on the basis of the first target area to obtain the second target area, performs edge detection on the first target area, obtains the first target contour, and performs motion estimation on the target image based on the first target contour to obtain the second target contour, and then segments the second target area based on the second target contour to obtain the target image to be recognized, and recognizes the target image to obtain the target image recognition result. The embodiment of the present invention is applied, and after performing color space recognition and image positioning on the image to be recognized, inter-class variance segmentation with an adaptive threshold based on the first target area is performed, so as to reduce the influence of background pixels on the inter-class variance segmentation, so as to make the result of the inter-class variance segmentation more accurate. At the same time, the inter-class variance segmentation based on the adaptive threshold is more targeted than the inter-class variance segmentation based on the same threshold, and further improves the accuracy of the inter-class variance segmentation, thereby improving the accuracy of image recognition. In addition, after determining the first target contour, performing motion estimation on the target image can better and more accurately obtain the contour of the moving target, improve the accuracy of the obtained target image, thereby improving the accuracy of image recognition and having a wider range of applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0096] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.
[0097] Figure 1 A schematic diagram of a flow chart of a first embodiment of an image recognition method provided in an embodiment of the present invention;
[0098] Figure 2 A schematic diagram of a flow chart of a second embodiment of an image recognition method provided in an embodiment of the present invention;
[0099] Figure 3 A schematic diagram of a process for determining a first target area in an embodiment of the present invention;
[0100] Figure 4 This is an example diagram for determining the maximum center point in an embodiment of the present invention;
[0101] Figure 5 A schematic diagram of a process for determining a second target area in an embodiment of the present invention;
[0102] Figure 6 A schematic diagram of a process for determining a second target profile in an embodiment of the present invention;
[0103] Figure 7 A schematic diagram of a flow chart of a third embodiment of an image recognition method provided in an embodiment of the present invention;
[0104] Figure 8 A schematic diagram of a first structure of an image recognition device provided in an embodiment of the present invention;
[0105] Fig. 9 A second structural schematic diagram of the image recognition device provided in an embodiment of the present invention;
[0106] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0107] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0108] The present invention provides an image recognition method, device, electronic device and storage medium. The image recognition method provided by the present invention is first exemplarily described below.
[0109] The image recognition method provided by the present invention can be used in scenarios such as video image recognition, picture intelligent recognition, etc.
[0110] The image recognition method provided by the present invention can be applied to electronic devices. The above electronic devices may include a client, a server, etc., which is not specifically limited by the present invention.
[0111] See also Figure 1 , Figure 1 is a schematic diagram of a first embodiment of an image recognition method provided by the present invention, the method may include the following steps:
[0112] Step S110, obtaining a picture to be identified.
[0113] Step S120, performing color space recognition and image positioning on the image to be recognized to obtain a first target area.
[0114] Step S130 , performing inter-class variance segmentation based on an adaptive threshold on the first target region to obtain a second target region.
[0115] Step S140: performing edge detection on the first target area to obtain a first target contour.
[0116] Step S150: Based on the first target contour, motion estimation is performed on the target image to obtain a second target contour.
[0117] Step S160, performing image segmentation on the second target area based on the second target contour to obtain a target image to be identified;
[0118] Step S170, performing image recognition on the target image to be recognized to obtain a target image recognition result.
[0119] In the embodiment of the present invention, the execution order of steps S130 and S140 is not specifically limited.
[0120] In an embodiment of the present invention, after obtaining a picture to be identified, color space identification and image positioning are performed on the picture to be identified, a first target area is obtained, and based on the first target area, inter-class variance segmentation with an adaptive threshold is performed to obtain a second target area, edge detection is performed on the first target area, a first target contour is obtained, and motion estimation is performed on the target image based on the first target contour to obtain a second target contour, and then the second target area is segmented based on the second target contour to obtain a target image to be identified, and the target image is identified to obtain a target image recognition result. By applying an embodiment of the present invention, after color space identification and image positioning are performed on the picture to be identified, inter-class variance segmentation with an adaptive threshold is performed based on the first target area, the influence of background pixels on the inter-class variance segmentation is reduced, so that the result of the inter-class variance segmentation is more accurate. At the same time, the inter-class variance segmentation based on the adaptive threshold is more targeted than the inter-class variance segmentation based on the same threshold, and the accuracy of the inter-class variance segmentation is further improved, thereby improving the accuracy of image recognition. In addition, after determining the first target contour, performing motion estimation on the target image can better and more accurately obtain the contour of the moving target, improve the accuracy of the obtained target image, thereby improving the accuracy of image recognition and having a wider range of applications.
[0121] The above steps S110-S170 are exemplarily described below:
[0122] The above-obtained image to be identified may be a screenshot of a video frame obtained by a user performing a screenshot operation while watching a video, or may be an image uploaded by a user to the Internet for intelligent identification, etc. The present invention does not make specific limitations on this.
[0123] After receiving the above-mentioned picture to be identified, the electronic device may first perform color space identification and image positioning on the picture to be identified to obtain the first target area.
[0124] It is understandable that the pixels belonging to the same target image (such as a dog, a person, or a car, etc.) have certain similarities. Image localization is to combine the image features of the parts of the image to be identified (such as color features, texture features, size, spatial relationship, etc.) to determine the location of the target image. Usually, the result of image localization is a detection frame of a preset shape (such as a square, a circle, etc.), which includes the target image.
[0125] Therefore, the first target area obtained by performing image positioning on the image to be identified is the approximate range of the target image to be identified. It may include the target image to be identified and part of the background. When the target image features are clearly distinguished from the background features, and the shape is the same as the shape of the preset detection frame, the above-mentioned first target area may also only include the target image to be identified.
[0126] In one embodiment of the present invention, based on Figure 1 ,like Figure 2 As shown, the first target area can be obtained by the following steps:
[0127] Step S121, performing color space recognition on the image to be recognized to obtain multiple color regions;
[0128] Step S122, extracting features from each color region to obtain features of each color region;
[0129] Step S123: merging the color regions based on the features of the color regions to obtain a first target region.
[0130] The above steps S121 to S123 are exemplarily described below:
[0131] It is understandable that by identifying the image to be identified based on different color spaces, the color of the image to be identified can be described from different aspects. For example, by identifying the image to be identified based on the RGB (red, green, blue) color space, the value of each pixel in the three color channels of red, green and blue can be obtained. By identifying the image to be identified based on the HSV (Hue, Saturation, Value) color space, the value of each pixel in the three channels of hue, saturation and brightness can be obtained. By identifying the image to be identified based on the Ycbcr (luminance represents brightness, CB and CR are the concentration offset components of blue and red) color space, the value of each pixel in these three channels can be obtained.
[0132] In this embodiment, the image to be identified can be identified based on a color space, for example, the image to be identified can be identified based on the HSV or YcbCr color space. Of course, in order to more comprehensively represent the color of the image to be identified, the image to be identified can also be identified based on multiple color spaces. For example, the image to be identified can be identified based on the HSV and YcbCr color spaces.
[0133] It is understandable that due to the influence of factors such as saturation, brightness, texture, exposure, etc., the target image (object, person, scenery, etc.) of the same color will appear in different colors in the picture. Therefore, based on different color spaces (such as HSV and YcbCr color spaces), after color space recognition of the image to be recognized, multiple color regions and the hue, saturation, brightness, etc. of each color region can be obtained.
[0134] After color space recognition is performed on the image to be recognized and each color region is obtained, features can be extracted for each color region. For example, color features, texture features, size, spatial relationship, and other features of each color region can be extracted. Next, the method of extracting color features, texture features, size, spatial relationship, and other features of each color region in an embodiment of the present invention is exemplified:
[0135] In the embodiment of the present invention, for each color region, a color histogram of the color region can be obtained based on the recognition result of the color region in the HSV color space as the color feature of the color region. For example, the color space can be divided into a number of small color intervals, each small interval becomes a bin of the histogram, and then, by calculating the number of pixels in the color region whose color falls within each small interval, a color histogram can be obtained as the color feature of the color region.
[0136] The texture feature describes the slowly changing or periodically changing surface structure and organization properties of the object surface. In this embodiment, the texture features of each color region can be obtained using the spatial gray-level co-occurrence matrix method (GLCM), or the LBP operator can be used to describe the texture features of each color region. The present invention does not make specific limitations on this.
[0137] Usually, in the process of image positioning, the scale of the merging operation of each part in the image to be identified needs to be relatively uniform to avoid merging an overly large area with an overly small area. Therefore, in an embodiment of the present invention, the size features of each color area can be extracted to achieve a better merging effect. In this embodiment, the size features of each color area can be represented by the number of pixels contained in each color area. At the same time, in order to make the scale of the merging operation relatively uniform, a weight can be set for each color area, and a larger weight can be set for a color area with a smaller size to increase the similarity in size between the smaller-scale area and other areas, so as to avoid merging smaller-sized areas with larger-sized areas that have been merged.
[0138] During the image positioning process, when merging images, if area A is contained in area B, then areas A and B need to be merged first. If area A and area B are difficult to connect, then areas A and B should not be merged. Therefore, it is necessary to calculate the spatial relationship features between the color areas to determine which areas can be merged and which areas cannot be merged in the subsequent merging process. As a specific implementation method, for any two color areas, the size of the bounding box after the two color areas are merged can be calculated as the spatial relationship feature between the two color areas. The smaller the bounding box, the more consistent the two color areas are, and the greater the probability that they can be merged.
[0139] Of course, in the present invention, the color features, texture features, size or spatial relationship features of the above-mentioned color regions may also be obtained by other feasible methods, and the present invention does not make any specific limitation on this.
[0140] When merging similar parts in the image to be identified based on the features of each color region, two color regions can be merged according to the color feature similarity, texture similarity, size feature similarity between the color regions, and the spatial relationship feature between the color regions. The above-mentioned feature similarities can be the Euclidean distance, cosine distance, etc. between the corresponding features of the two color regions, and the present invention does not specifically limit this.
[0141] As an implementation method, a color feature similarity threshold, a texture feature similarity threshold, a size feature similarity threshold, and a spatial relationship feature threshold (such as the size threshold of the above-mentioned bounding box) can be preset, and two color regions whose feature similarities exceed the corresponding similarity thresholds are merged. The above-mentioned similarity thresholds can be the same or different, and the present invention does not specifically limit this. Of course, the similarity between the two color regions can also be calculated based on the above-mentioned color feature similarity, texture feature similarity, size feature similarity, and spatial relationship feature, and the color regions can be merged based on the similarity. At the same time, a weight can also be set for each feature to characterize the importance of each feature.
[0142] For example, for any two color regions r i , r j , the color feature similarity between the two color regions is recorded as Scolor ij , with a weight of a1; the texture feature similarity between the two color regions is recorded as Stexture ij , with a weight of a2; the size feature similarity between the two color regions is recorded as Ssize ij , with a weight of a3; the spatial relationship feature between the two color regions is recorded as Sfill ij , with a weight of a4. Then the similarity S between the two color regions is a1*Scolor ij+a2*Stexture ij +a3*Ssize ij +a4*Sfill ij If the similarity S is greater than the preset similarity threshold, the two color regions can be merged, otherwise they cannot.
[0143] As an implementation method, color region merging of the image to be identified can be achieved through the following steps:
[0144] 1. Initialize the similarity measure set S and the region set R.
[0145] 2. Traverse all color regions and obtain all adjacent color region pairs (r i ,r j )gather.
[0146] 3. Traverse all color region pairs (r i ,r j ) set, calculate the region r i and region r j , and add this similarity measure to S.
[0147] 4. If S is not empty, loop processing is performed until all regions are added to R: In the loop, first find the corresponding region pair with the greatest similarity (r i ,r j ). Then the area r i and region r j Merge and r i , r j Add R. The merged area is recorded as r t . Then delete the region r from S i Similarity measure of other adjacent regions and region r j The similarity measure of adjacent regions. After that, the region r is calculated t The similarity between it and its adjacent regions is measured and added to S.
[0148] After merging the color regions, the target image detection frame can be determined as the first target region.
[0149] As a specific implementation of the embodiment of the present invention, Figure 3 As shown, the above step S123, based on the characteristics of the color regions, merges the color regions to obtain the first target region, and before that, the following steps may also be included:
[0150] Step S310, performing color space recognition on the image to be recognized, and determining the center point of each color area.
[0151] In this embodiment, the center point of each color region may be determined based on the number of pixels contained in each color region and the position of each pixel.
[0152] Step S320: For each color region center point, calculate the distance from the center point to the center points of other color regions.
[0153] Step S330, based on the distances from the center points of each color area to the center points of other color areas, determining, among the center points of each color area, a maximum center point corresponding to the maximum distance and a minimum center point corresponding to the minimum distance.
[0154] Step S340, determining a target image candidate region based on the maximum center point and the minimum center point;
[0155] The above step S123, merging the color regions based on the characteristics of the color regions to obtain the first target region, can be achieved by the following steps:
[0156] Step S350: Based on the characteristics of the target image candidate region and the color regions, the color regions are merged to obtain a first target region.
[0157] The following is an exemplary description of steps S320 to S350:
[0158] In this embodiment, the maximum center point and the minimum center point may be found by an iterative method.
[0159] like Figure 4 As shown, taking finding the maximum center point as an example, the process of finding the maximum center point and the minimum center point in the present invention is described:
[0160] Step S401, determining the center point of any color area as the maximum center point.
[0161] Step S402, calculating the distance between the maximum center point and the center points of other color regions.
[0162] Step S403, determining the center point with the largest distance from the maximum center point as the new maximum center point, and recording the maximum distance at this time.
[0163] In this step, the position of the new maximum center point may also be recorded.
[0164] Step S404, calculating the distance between the new maximum center point and the center points of other color regions.
[0165] Step S405, determining whether the maximum distance between the center point and the center points of other color regions is greater than the recorded maximum distance. If so, executing step S406; if not, executing step S407.
[0166] If they are equal, step S406 may also be executed, and the new center point and the center points of other color regions with the largest distance from the new center point are used as the maximum center point to update the previously recorded maximum center point information.
[0167] Step S406, taking the center point of the other color area with the largest distance from the new center point as the new maximum center point, and returning to step S404.
[0168] Step S407, reselect any color region center point that has not been used as the maximum center point, and calculate the distance from the center point to the center points of other color regions. Return to step S405.
[0169] In this embodiment, after the distances between the center point of each color area and the center points of other color areas are calculated, the maximum center point recorded at this time can be used as the maximum center point found.
[0170] Of course, the distances between the center points of each color region and the center points of other color regions can also be calculated by traversal, and the center point corresponding to the maximum distance among these distances is taken as the maximum center point.
[0171] The process of finding the minimum center point can refer to the process of finding the maximum center point mentioned above, which will not be repeated here.
[0172] After determining the maximum center point and the minimum center point, the target image candidate area can be determined based on the range from the maximum center point to the minimum center point. Of course, all the maximum center points and the minimum center points generated in the iterative search for the maximum center point and the minimum center point can also be stored, and the range of the target image candidate area can be determined based on the range from the center of each maximum center point to the center of each minimum center point.
[0173] Through the above steps S310 to S350, the approximate range of the target image can be determined in the picture to be identified. The target image candidate area may include the complete target image, a part of the target image, or a part of the background image. Compared with the picture to be identified, the target image candidate area reduces the influence of the background image on the target image to a certain extent, and improves the accuracy of subsequent image segmentation.
[0174] Afterwards, similar image blocks may be merged based on the determined target image candidate region and the features of each color region to obtain the first target region.
[0175] After obtaining the first target region, inter-class variance segmentation may be performed on the first target region to segment the target image (foreground image) from the background image, thereby obtaining a second target image region containing the target image.
[0176] In one embodiment of the present disclosure, Figure 5 As shown, the following steps may be used to obtain the second target area:
[0177] Step S131, obtaining the grayscale features of the first target area.
[0178] In this embodiment, the grayscale feature of the first target area can be represented by a grayscale histogram. It can be understood that the first target area is composed of pixels with different grayscale values, and the grayscale distribution in the first target area is an important feature of the image. The grayscale histogram of the first target area can describe the grayscale distribution in the first target area, and can intuitively show how much each grayscale level occupies in the image. The horizontal coordinate of the grayscale histogram is the grayscale level, and the vertical coordinate is the frequency of occurrence of the grayscale level.
[0179] Of course, the grayscale feature of the first target area may also be the grayscale average value, grayscale variance, energy, entropy, grayscale contrast amplitude, etc. of the first target area, and the present invention does not make any specific limitation on this.
[0180] Step S132: acquiring the grayscale probability distribution of each pixel in the first target area based on the grayscale feature of the first target area.
[0181] As a specific implementation, the gamma probability distribution of the grayscale of each pixel in the first target area may be determined based on the grayscale feature of the first target area.
[0182] As a specific implementation method, the first target area can be firstly smoothed by Gaussian filtering, and then the gradient image can be obtained by using Roberts crossover operator. According to this trapezoidal image, combined with the hist of the grayscale histogram, the Gamma probability distribution function is calculated. Gaussian filtering is a linear smoothing filter. By performing Gaussian filtering on the first target area, noise reduction of the first target area can be achieved.
[0183] By calculating the gamma probability distribution of each pixel in the first target area, it is possible to prevent the bleached (overexposed) or overly dark (underexposed) area in the first target area from affecting image recognition.
[0184] Step S133: determining a membership function of the grayscale of each pixel in the first target area based on the grayscale probability distribution of each pixel in the first target area.
[0185] In this embodiment, the membership function may be determined based on a gamma probability distribution function of the grayscale of each pixel in the first target area.
[0186] In the embodiment of the present invention, the grayscale information of each pixel in the first target area is first scanned to determine the maximum value, minimum value and average value of each grayscale pixel in the first target area. According to the segmentation result of the initial threshold, the grayscale pixels less than the threshold are used as the background, and the grayscale pixels greater than the threshold are used as the foreground. The above initial threshold can be set manually.
[0187] As a specific implementation, the above membership function grainf may be:
[0188] grainf=1-c*image(I,j)-average(I,j)+max.
[0189] In this formula, c is a normalization factor, which can be preset manually, for example, 0.3, image(I,j) is the position of the image, average(I,j) is the average grayscale value of the image, and max is the normalized maximum grayscale value of the image.
[0190] Step S134, for each pixel in the first target area, based on the divided multiple granularity layer spaces, calculating the similarity between the pixel and its adjacent pixels;
[0191] In this embodiment, the first target area can be divided into different granularity layers by using different density sampling or calculating the average pixel value of different size areas. Then, the similarity between each pixel in the first target area and the surrounding pixels can be calculated based on the different granularity layers. For example, the similarity between each pixel and the surrounding 4 or 8 pixels can be calculated.
[0192] Step S135, for each pixel in the first target area, based on the membership function and the similarity between the pixel and its adjacent pixels, obtaining the membership of the pixel to the foreground image;
[0193] As described above, the similarity between each pixel and its adjacent pixels is calculated in multiple granularity layers. Therefore, when calculating the degree of membership, for each pixel, the average similarity between it and its adjacent pixels in multiple granularity layers can be used as the similarity between the pixel and its neighboring pixels.
[0194] As a specific implementation, the pixel value f at each position of the first target area and the pixel number histogram corresponding to the pixel value can be extracted, and on this basis, the mean value U0 of the foreground and the mean value U1 of the background are obtained, and finally the membership is calculated in combination with the normalization factor C. Specifically, the membership of each pixel belonging to the foreground image can be calculated by the following steps:
[0195] ①. Calculate the average value of foreground pixels:
[0196] U0=Σ([x(i,j)*hist(f(i,j))]] / Σ(hist(f(i,j)))
[0197] Where x(i,j) is the image pixel position, hist(f(i,j)) represents the grayscale histogram of each pixel in the first target area, and c is the normalization factor.
[0198] ②. Calculate the grayscale information value U:
[0199] U=exp(-c*x(i,j)-U 0)
[0200] ③. Calculate spatial neighborhood similarity:
[0201] w=1-Σ[σ(x(i+1,j+1)-x(i,j))]
[0202] Among them, σ represents the similarity weight between the pixels in the window range of the (i+1)th position and the (i)th position and the central pixel. This value can be preset manually.
[0203] ④. Calculate the membership of the image:
[0204] d=Σ[x(i,j)*n / U0] / (w*U)
[0205] Where n is the number of neighborhood pixels of the preset window. The window can be set to include 4 neighborhood pixels, 8 neighborhood pixels, etc.
[0206] The membership function is a FDM (Finite Differential Method) function. Because most image threshold segmentation algorithms only consider the gray level value of the image and ignore the spatial neighborhood information of the pixel, the proportion of incorrect segmentation between the target and the background is increased. In the embodiment of the present invention, after introducing the FDM algorithm, a better image segmentation effect can be achieved.
[0207] Step S136: Based on the degree of membership of each pixel belonging to the foreground image, perform inter-class variance segmentation on the first target region to obtain a second target region.
[0208] As a specific implementation, the pixels belonging to the foreground image whose membership degree is greater than a preset membership degree threshold can be used as foreground pixels, and the rest can be used as background pixels. The foreground pixels constitute the second target area.
[0209] In the embodiment of the present invention, the first target contour of the target image can be extracted using the canny edge detection algorithm based on the first target area. Of course, other feasible edge detection algorithms can also be used to extract the first target contour of the target image, and the present invention does not specifically limit this.
[0210] In some images to be identified, the target image is not static (such as people and objects in motion). In an embodiment of the present invention, an improved TSS algorithm is used to achieve segmentation and extraction of these images that move in translation and forward and backward motion, thereby enabling more efficient segmentation processing.
[0211] In one embodiment of the present disclosure, Figure 6 As shown, the following steps may be performed for each target pixel point on the first target contour to obtain the second target contour:
[0212] Step S601 : for each target pixel on the first target contour, calculate a first mean square error and a first minimum SAD value of the target pixel and a first preset number of pixels within a first preset range.
[0213] The first preset range and the first preset number can be considered to be preset. For example, the target pixel can be taken as the center, 6 can be used as the step length, and 8 points around the target pixel can be selected to calculate the mean square error of 9 pixels including the target pixel. Of course, the first preset range can also be determined with a step length of 7, 8, etc., and the first preset number can also be 12, 16, etc., which is not specifically limited in the present invention.
[0214] The first minimum SAD value (Sum of Absolute Differences, absolute value of differences) can be obtained by respectively calculating the SAD value of each pixel point for the target pixel point and a first preset number of pixels around it, and determining the minimum SAD value therein as the first minimum SAD value.
[0215] As a specific implementation, it can be:
[0216] 1. Create a convolution window, the initial size of which can be manually defined, for example, 1 / 2 or 1 / 3 of the step size of the first preset range.
[0217] 2. Determine the target matching point of the target pixel.
[0218] In the embodiment of the present invention, the target matching point of the target pixel point can be determined by the following steps:
[0219] (1) Initialize a target image window. begin is the position coordinate information of the image that is initially matched. According to the moving range D, determine the current end position coordinate information end. The above moving range D can be preset manually, such as 25 pixels. The position coordinate information of the image that is initially matched can be the first target contour. (2) Find the position (x, y) of the corresponding pixel point in begin, use this position as the anchor point, and determine the current region regionalLeft by the radiation of begin. (3) Take end as the starting point of the match, recorded as (m, n), and determine the current region regionalRight by the radiation of end. (4) During the convolution movement process, continuously calculate temp = regionalRight (m, n) - regionalLeft (x, y), and record the results in the det matrix. (5) The minimum value array of temp in each row of the det matrix is located at the matching point area of the target pixel.
[0220] The following uses the radiation of begin as an example to illustrate how to determine the current region regionleft:
[0221] The pixel point indexed by the begin position is a matrix expression, which is equivalent to the quadrant arrangement of the plane rectangular coordinate system: GrayPic = imhist (rgb2gray (picture));
[0222] image=figure(GrayPic)
[0223] The radiation of begin determines the current region regionalLeft by 2D coordinates, where:
[0224] X = subplot(image.hist_x)
[0225] Y = subplot(image.hist_y)
[0226] Therefore, regionalLeft is the current image information determined by the starting position X and Y.
[0227] 3. Cover each pixel to be found within the first preset range with the above convolution window, and calculate the sum of the grayscale values in the window. For each pixel to be found, when the convolution window covers the pixel, the absolute value of the change in the grayscale value is calculated during the movement of the convolution window to obtain the SAD value of the pixel.
[0228] 4. Determine the minimum SAD value among all the pixels to be determined as the first minimum SAD value.
[0229] Step S602, determining the direction of the pixel point with the smallest first mean square error as the first minimum distortion direction;
[0230] In this embodiment, each step of searching is performed in the direction of minimum distortion.
[0231] Step S603, determining a pixel point within the first preset range and in the first minimum distortion direction as a second search starting point;
[0232] Step S604, calculating the second mean square error and the second minimum SAD value of the second search starting point and a second preset number of pixel points within a second preset range;
[0233] The step length of the second preset range may be 1 / 2 of the step length of the first preset range. The second preset number may be the same as or different from the first preset number.
[0234] Step S605, determining the direction of the pixel point with the smallest second mean square error as the second minimum distortion direction;
[0235] Step S606: determine a pixel point within the second preset range and in the second minimum distortion direction as a third search starting point.
[0236] Step S607, calculating the third search starting point and a third minimum SAD value among a third preset number of pixel points within a third preset range.
[0237] The step length of the third preset range may be 1 / 2 of the step length of the second preset range. The third preset number may be the same as or different from the second preset number.
[0238] In a specific embodiment of the present invention, the mean square error of 9 pixels including the target pixel may be calculated each time, and after the minimum distortion direction is determined, the search step size is halved and the next search is performed until the search is completed.
[0239] The specific processing process of the above steps S603-S607 can refer to steps S601-S602, which will not be repeated here, and only a brief explanation will be given.
[0240] Step S608: Determine the pixel point corresponding to the target minimum SAD value based on the first minimum SAD value, the second minimum SAD value and the third minimum SAD value.
[0241] Step S609, calculating the target grayscale value corresponding to the target pixel point based on the grayscale value of the pixel point corresponding to the target minimum SAD value and the average grayscale value of the image to be identified.
[0242] In this embodiment, the grayscale value of the pixel corresponding to the target minimum SAD value and the weighted difference between the average grayscale value of the image to be identified can be used as the target grayscale value corresponding to the target pixel. The weight of the above weighted difference can be Si = (bi-ai) / max (ai, bi), where ai refers to the pixel value corresponding to the target minimum SAD value. Bi refers to the average pixel value of the entire image.
[0243] In the embodiment of the present invention, an ai value is determined in each step of the search, and this ai value is used as the central pixel value of the current search area. However, because the convolution window will also move to other search areas through the step size, the SAD value of the previous area may not be the minimum SAD value of the entire convolution window. Therefore, it is necessary to calculate through round-robin, that is, the calculation of each step of the movement of the convolution window, combined with the mean square error and search range mentioned above, and finally calculate the ai value corresponding to the minimum SAD value.
[0244] In practical applications, the improved TSS algorithm also has better processing effect and faster processing speed, and has less calculation amount.
[0245] In the embodiment of the present invention, the above steps S601 to S609 may be calculated for each color space respectively, and the target grayscale value corresponding to the target pixel point corresponding to each color space is obtained.
[0246] Then, for a target pixel, an average value or a weighted average value of the target grayscale values corresponding to the target pixel in each color space may be taken as the target grayscale value of the target pixel.
[0247] Step S610: obtaining a second target contour based on the target grayscale value of each target pixel point on the first target contour.
[0248] After executing the above steps S601 to S610 for each pixel point on the first target contour, an updated target contour can be obtained as the second target contour.
[0249] After the second target contour is obtained, the second target area can be segmented based on the second target contour to obtain the target image to be identified.
[0250] In the embodiment of the present invention, the segmentation of the second target area to obtain the target image to be identified can be achieved by relying on a pre-trained image segmentation model.
[0251] The structure of the above model can be ResNet+FNN. When using this model for image segmentation, upsampling is first performed to expand the pixels, and then downsampling is performed to obtain the segmented image. In this embodiment, upsampling is used to reduce the loss of some information, thereby obtaining a more accurate segmentation boundary, a deep network that can cover a larger part of the context information, and the image information obtained is also more characteristic.
[0252] When training the model, you can first use uniform float format data to render the image edge. The uniform float format data can include texOffsetX (horizontal) and texOffsetY (vertical); use vec3 data (vec3(pow(gl_FragCoord.xy,exp)) to calculate the spatial position, and use the Prewitt edge detection method (texture3D(colorAttachTex,gl_FragCoord.xy / texSize+vec3(texOffsetX,texOffsetY)) to calculate the image edge. Then input the above image data into the model, and perform Conv2d (convolution in two dimensions), Batch Norm2d normalization processing and ReLU transformation and extraction are performed to obtain the output result. During the training process, the parameters of each layer in the model can be adjusted according to the output results. In the output result, a Prewitt operator prewitt is defined, and changeX+=texOffsetX[i]*prewitt_x[i]; changeY+=texOffsetY[i]*prewitt_y[i] are set. The data parameters of each layer in the output result model are dynamically adjusted through changeX and changeY, and gl_Position rendering is performed. The loss function can also be preset. When the value of the loss function calculated based on the model output result and the input image is less than the preset threshold, the training is stopped.
[0253] In one embodiment of the present disclosure, based on Figure 1 ,like Figure 7 As shown, after obtaining the target image recognition result, the following steps may also be included:
[0254] Step S780: Based on the target image recognition result, the associated information preset for the recognition result is displayed.
[0255] In the embodiment of the present invention, a recognition result library may be preset, which may include various recognition results (such as characters, objects, etc.) and associated information of each recognition result (such as a brief introduction of the character, the works he participated in, links to the object, etc.).
[0256] When the target image is recognized, the target image can be recognized based on the preset recognition result library. If the target image is recognized, the associated information associated with the recognition result of the target image can be displayed. For example, it can be displayed in the sidebar of the video page, or next to the target image, etc., and the present invention does not specifically limit this.
[0257] In one embodiment of the present disclosure, based on Figure 1 ,like Figure 7 As shown, before performing inter-class variance segmentation based on an adaptive threshold on the first target area, the following steps may also be included:
[0258] Step S730: performing gradient adjustment on each pixel in the first target area to obtain an adjusted first target area.
[0259] Because images have many texture features, as mentioned above, in the embodiment of the present invention, the LBP operator can be used to characterize the texture features of the image to be identified. In order to better describe the features of this image, it is necessary to edit the gradient function through horizontal, vertical and diagonal wavelet transforms. And adjust the first target area based on the gradient function to ensure the rationality of the texture information. The above gradient function can be set manually, and the present invention does not make specific limitations on this.
[0260] As a specific implementation, the above-mentioned gradient function can be obtained based on the Gabor function and the LBP operator. Among them, the LBP operator represents the texture features of the image to be identified, and the Gabor function can be used for edge detection and texture feature extraction. Specifically, the first target area can be gradient adjusted by the Gabor function. Among them, the height and width of the Gabor window and the adjustment angle can be preset. The gradient amplitude map and phase map are obtained by the above-mentioned Gabor function. In practical applications, the above-mentioned Gabor function can be recorded as filteredImage(I,h,w,theta). Among them, I is the input image, h and w are the height and width of the Gabor window, and theta corresponds to the adjustment angle.
[0261] In the process of edge detection, because there may be a lack of edge information at the boundary connection of the target area, the fusion accuracy of some edges may be low. Therefore, in the actual gradient adjustment, the coordinates of the input image can also be adjusted, and the adjusted input image can be gradient adjusted. As a specific implementation, the x-coordinate and y-coordinate of each pixel can be adjusted by the following formulas: x = x*cos(theta) + y*sin(theta), y = x*sin(theta) + y*cos(theta).
[0262] Correspondingly, the above gradient adjustment function can be recorded as filteredImage(I,h,w,theta,x,y). In this way, by introducing the coordinates of the filter adjustment variables x and y, the final gradient function is made more accurate.
[0263] Correspondingly, such as Figure 7 As shown, the above step of performing inter-class variance segmentation on the first target area based on the adaptive threshold to obtain the second target area may include:
[0264] Step S740 , performing inter-class variance segmentation based on an adaptive threshold on the adjusted first target region to obtain a second target region.
[0265] In another aspect of the embodiment of the present invention, an image recognition device is also provided. Figure 8 As shown, the device may include:
[0266] The to-be-recognized picture acquisition module 810 is used to acquire the to-be-recognized picture;
[0267] A first target region acquisition module 820 is used to perform color space recognition and image positioning on the image to be identified to obtain a first target region; wherein the first target region includes the target image to be identified;
[0268] A second target region acquisition module 830 is used to perform inter-class variance segmentation based on an adaptive threshold on the first target region to obtain a second target region;
[0269] A first target contour acquisition module 840 is used to perform edge detection on the first target area to acquire a first target contour;
[0270] A second target contour acquisition module 850 is used to perform motion estimation on the target image based on the first target contour to obtain a second target contour;
[0271] The target image acquisition module 860 is used to perform image segmentation on the second target area based on the second target contour to obtain the target image to be identified;
[0272] The image recognition module 870 is used to perform image recognition on the target image to be recognized to obtain a target image recognition result.
[0273] In an embodiment of the present invention, after obtaining a picture to be identified, color space identification and image positioning are performed on the picture to be identified, a first target area is obtained, and based on the first target area, inter-class variance segmentation with an adaptive threshold is performed to obtain a second target area, edge detection is performed on the first target area, a first target contour is obtained, and motion estimation is performed on the target image based on the first target contour to obtain a second target contour, and then the second target area is segmented based on the second target contour to obtain a target image to be identified, and the target image is identified to obtain a target image recognition result. By applying an embodiment of the present invention, after color space identification and image positioning are performed on the picture to be identified, inter-class variance segmentation with an adaptive threshold is performed based on the first target area, the influence of background pixels on the inter-class variance segmentation is reduced, so that the result of the inter-class variance segmentation is more accurate. At the same time, the inter-class variance segmentation based on the adaptive threshold is more targeted than the inter-class variance segmentation based on the same threshold, and the accuracy of the inter-class variance segmentation is further improved, thereby improving the accuracy of image recognition. In addition, after determining the first target contour, performing motion estimation on the target image can better and more accurately obtain the contour of the moving target, improve the accuracy of the obtained target image, thereby improving the accuracy of image recognition and having a wider range of applications.
[0274] In one embodiment of the present invention, the first target area acquisition module is used to:
[0275] Performing color space recognition on the image to be recognized to obtain multiple color regions;
[0276] Feature extraction is performed on each color region to obtain features of each color region.
[0277] Based on the features of the color regions, the color regions are merged to obtain a first target region.
[0278] In an embodiment of the present invention, before merging the color regions based on the features of the color regions to obtain the first target region, the method further includes:
[0279] Performing color space recognition on the image to be recognized to determine the center point of each color area;
[0280] For each color region center point, calculate the distance from the center point to the center points of other color regions;
[0281] Based on the distances from the center points of each color area to the center points of other color areas, determining, among the center points of each color area, a maximum center point corresponding to the maximum distance and a minimum center point corresponding to the minimum distance;
[0282] Determine a target image candidate area based on the maximum center point and the minimum center point;
[0283] The step of merging the color regions based on the features of the color regions to obtain a first target region includes:
[0284] Based on the characteristics of the target image candidate region and the color regions, the color regions are merged to obtain a first target region.
[0285] In one embodiment of the present invention, the second target area acquisition module is used to:
[0286] Acquire the grayscale feature of the first target area;
[0287] Based on the grayscale feature of the first target area, obtaining the grayscale probability distribution of each pixel in the first target area;
[0288] Determining a membership function of the grayscale of each pixel in the first target area based on the grayscale probability distribution of each pixel in the first target area;
[0289] For each pixel in the first target area, based on the divided multiple granularity layer spaces, calculate the similarity between the pixel and its adjacent pixels;
[0290] For each pixel in the first target area, based on the membership function and the similarity between the pixel and its adjacent pixels, obtain the membership of the pixel to the foreground image;
[0291] Based on the membership degree of each pixel belonging to the foreground image, the first target region is segmented by inter-class variance to obtain a second target region.
[0292] In one embodiment of the present invention, the second target profile acquisition module is used to:
[0293] For each target pixel point on the first target contour, calculating a first mean square error and a first minimum SAD value of the target pixel point and a first preset number of pixel points within a first preset range;
[0294] Determine the direction of the pixel point with the smallest first mean square error as the first minimum distortion direction;
[0295] Determine a pixel point within the first preset range and in the first minimum distortion direction as a second search starting point;
[0296] Calculating the second search starting point, and a second mean square error and a second minimum SAD value of a second preset number of pixel points within a second preset range;
[0297] Determine the direction of the pixel point with the smallest second mean square error as the second minimum distortion direction;
[0298] Determine a pixel point within the second preset range and in the second minimum distortion direction as a third search starting point;
[0299] Calculating the third search starting point and a third minimum SAD value among a third preset number of pixel points within a third preset range;
[0300] Determine a pixel point corresponding to a target minimum SAD value based on the first minimum SAD value, the second minimum SAD value, and the third minimum SAD value;
[0301] Calculate the target grayscale value corresponding to the target pixel based on the grayscale value of the pixel corresponding to the target minimum SAD value and the average grayscale value of the image to be identified;
[0302] A second target contour is obtained based on the target grayscale value of each target pixel point on the first target contour.
[0303] In one embodiment of the present invention, Figure 8 On the basis of Fig. 9 As shown, the above device may also include:
[0304] A gradient adjustment module 930, configured to perform gradient adjustment on each pixel in the first target area to obtain an adjusted first target area;
[0305] The second target contour acquisition module is used to perform inter-class variance segmentation based on an adaptive threshold on the adjusted first target area to obtain a second target area.
[0306] In one embodiment of the present invention, based on Figure 8 ,like Fig. 9 As shown, the above device may also include:
[0307] The associated information display module 980 is used to display the associated information preset for the recognition result based on the target image recognition result.
[0308] The embodiment of the present invention further provides an electronic device, such as Fig.10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.
[0309] Memory 1003, used for storing computer programs;
[0310] The processor 1001 is used to execute the program stored in the memory 1003, and implements the following steps:
[0311] Get the image to be identified;
[0312] Performing color space recognition and image positioning on the image to be identified to obtain a first target area; wherein the first target area includes the target image to be identified;
[0313] Performing inter-class variance segmentation based on an adaptive threshold on the first target area to obtain a second target area;
[0314] Performing edge detection on the first target area to obtain a first target contour;
[0315] Based on the first target contour, performing motion estimation on the target image to obtain a second target contour;
[0316] Based on the second target contour, performing image segmentation on the second target area to obtain a target image to be identified;
[0317] Perform image recognition on the target image to be recognized to obtain a target image recognition result.
[0318] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0319] The communication interface is used for communication between the above terminal and other devices.
[0320] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0321] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0322] In another embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the image recognition method described in any one of the above embodiments is implemented.
[0323] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the image recognition method described in any one of the above embodiments.
[0324] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0325] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0326] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, storage medium and program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0327] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. An image recognition method, characterized in that: include: Get the image to be identified; Performing color space recognition and image positioning on the image to be identified to obtain a first target area; wherein the first target area includes the target image to be identified; Performing inter-class variance segmentation based on an adaptive threshold on the first target area to obtain a second target area; Performing edge detection on the first target area to obtain a first target contour; Based on the first target contour, performing motion estimation on the target image to obtain a second target contour; Based on the second target contour, performing image segmentation on the second target area to obtain a target image to be identified; Perform image recognition on the target image to be recognized to obtain a target image recognition result.
2. The method according to claim 1, characterized in that The step of performing color space recognition and image positioning on the image to be recognized to obtain the first target area includes: Performing color space recognition on the image to be recognized to obtain multiple color regions; Extracting features from each color region to obtain features of each color region; Based on the features of the color regions, the color regions are merged to obtain a first target region.
3. The method according to claim 2, characterized in that Before the step of merging the color regions based on the features of the color regions to obtain the first target region, the step further includes: Performing color space recognition on the image to be recognized to determine the center point of each color area; For each color region center point, calculate the distance from the center point to the center points of other color regions; Based on the distances from the center points of each color area to the center points of other color areas, determining, among the center points of each color area, a maximum center point corresponding to the maximum distance and a minimum center point corresponding to the minimum distance; Determine a target image candidate area based on the maximum center point and the minimum center point; The step of merging the color regions based on the features of the color regions to obtain a first target region includes: Based on the characteristics of the target image candidate region and the color regions, the color regions are merged to obtain a first target region.
4. The method according to claim 1, characterized in that: The step of performing inter-class variance segmentation on the first target area based on an adaptive threshold to obtain a second target area includes: Acquire the grayscale feature of the first target area; Based on the grayscale feature of the first target area, obtaining the grayscale probability distribution of each pixel in the first target area; Determining a membership function of the grayscale of each pixel in the first target area based on the grayscale probability distribution of each pixel in the first target area; For each pixel in the first target area, based on the divided multiple granularity layer spaces, calculate the similarity between the pixel and its adjacent pixels; For each pixel in the first target area, based on the membership function and the similarity between the pixel and its adjacent pixels, obtain the membership of the pixel to the foreground image; Based on the membership degree of each pixel belonging to the foreground image, the first target region is segmented by inter-class variance to obtain a second target region.
5. The method according to claim 1, characterized in that The step of performing motion estimation on the target image based on the first target contour to obtain the second target contour comprises: For each target pixel point on the first target contour, calculating a first mean square error and a first minimum SAD value of the target pixel point and a first preset number of pixel points within a first preset range; Determine the direction of the pixel point with the smallest first mean square error as the first minimum distortion direction; Determine a pixel point within the first preset range and in the first minimum distortion direction as a second search starting point; Calculating the second search starting point, and a second mean square error and a second minimum SAD value of a second preset number of pixel points within a second preset range; Determine the direction of the pixel point with the smallest second mean square error as the second minimum distortion direction; Determine a pixel point within the second preset range and in the second minimum distortion direction as a third search starting point; Calculating the third search starting point and a third minimum SAD value among a third preset number of pixel points within a third preset range; Determine a pixel point corresponding to a target minimum SAD value based on the first minimum SAD value, the second minimum SAD value, and the third minimum SAD value; Calculate the target grayscale value corresponding to the target pixel based on the grayscale value of the pixel corresponding to the target minimum SAD value and the average grayscale value of the image to be identified; A second target contour is obtained based on the target grayscale value of each target pixel point on the first target contour.
6. The method according to claim 1, characterized in that Before the step of performing inter-class variance segmentation based on an adaptive threshold on the first target area to obtain a second target area, the step further includes: Performing gradient adjustment on each pixel in the first target area to obtain an adjusted first target area; The step of performing inter-class variance segmentation on the first target area based on an adaptive threshold to obtain a second target area includes: The adjusted first target region is segmented based on inter-class variance of an adaptive threshold to obtain a second target region.
7. The method according to claim 1, characterized in that Also includes: Based on the target image recognition result, the associated information preset for the recognition result is displayed.
8. An image recognition device, characterized in that: include: The module for obtaining the image to be identified is used to obtain the image to be identified; A first target region acquisition module, used to perform color space recognition and image positioning on the image to be identified, to obtain a first target region; wherein the first target region includes the target image to be identified; A second target region acquisition module, configured to perform inter-class variance segmentation on the first target region based on an adaptive threshold to obtain a second target region; A first target contour acquisition module, used for performing edge detection on the first target area to acquire a first target contour; A second target contour acquisition module, configured to perform motion estimation on the target image based on the first target contour to obtain a second target contour; A module for acquiring an image of a target to be identified, configured to perform image segmentation on the second target area based on the second target contour to obtain an image of the target to be identified; The image recognition module is used to perform image recognition on the target image to be recognized to obtain a target image recognition result.
9. The device according to claim 8, characterized in that The first target area acquisition module is used to: Performing color space recognition on the image to be recognized to obtain multiple color regions; Extracting features from each color region to obtain features of each color region; Based on the features of the color regions, the color regions are merged to obtain a first target region.
10. The device according to claim 9, characterized in that Before merging the color regions based on the features of the color regions to obtain the first target region, the method further includes: Performing color space recognition on the image to be recognized to determine the center point of each color area; For each color region center point, calculate the distance from the center point to the center points of other color regions; Based on the distances from the center points of each color area to the center points of other color areas, determining, among the center points of each color area, a maximum center point corresponding to the maximum distance and a minimum center point corresponding to the minimum distance; Determine a target image candidate area based on the maximum center point and the minimum center point; The step of merging the color regions based on the features of the color regions to obtain a first target region includes: Based on the characteristics of the target image candidate region and the color regions, the color regions are merged to obtain a first target region.
11. The device according to claim 8, characterized in that The second target area acquisition module is used to: Acquire the grayscale feature of the first target area; Based on the grayscale feature of the first target area, obtaining the grayscale probability distribution of each pixel in the first target area; Determining a membership function of the grayscale of each pixel in the first target area based on the grayscale probability distribution of each pixel in the first target area; For each pixel in the first target area, based on the divided multiple granularity layer spaces, calculate the similarity between the pixel and its adjacent pixels; For each pixel in the first target area, based on the membership function and the similarity between the pixel and its adjacent pixels, obtain the membership of the pixel to the foreground image; Based on the membership degree of each pixel belonging to the foreground image, the first target region is segmented by inter-class variance to obtain a second target region.
12. The device according to claim 8, characterized in that The second target profile acquisition module is used to: For each target pixel point on the first target contour, calculating a first mean square error and a first minimum SAD value of the target pixel point and a first preset number of pixel points within a first preset range; Determine the direction of the pixel point with the smallest first mean square error as the first minimum distortion direction; Determine a pixel point within the first preset range and in the first minimum distortion direction as a second search starting point; Calculating the second search starting point, and a second mean square error and a second minimum SAD value of a second preset number of pixel points within a second preset range; Determine the direction of the pixel point with the smallest second mean square error as the second minimum distortion direction; Determine a pixel point within the second preset range and in the second minimum distortion direction as a third search starting point; Calculating the third search starting point and a third minimum SAD value among a third preset number of pixel points within a third preset range; Determine a pixel point corresponding to a target minimum SAD value based on the first minimum SAD value, the second minimum SAD value, and the third minimum SAD value; Calculate the target grayscale value corresponding to the target pixel based on the grayscale value of the pixel corresponding to the target minimum SAD value and the average grayscale value of the image to be identified; A second target contour is obtained based on the target grayscale value of each target pixel point on the first target contour.
13. The device according to claim 8, characterized in that The device further comprises: A gradient adjustment module, used for performing gradient adjustment on each pixel in the first target area to obtain an adjusted first target area; The second target contour acquisition module is used to perform inter-class variance segmentation based on an adaptive threshold on the adjusted first target area to obtain a second target area.
14. The device according to claim 8, characterized in that Also includes: The associated information display module is used to display the associated information preset for the recognition result based on the target image recognition result.
15. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.
16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.
