Image processing method and device

Abstract

The present disclosure relates to technology for processing an image, and provides an image processing device comprising: an object identification unit for distinguishing and identifying a plurality of objects included in an image; and an object sorting unit for filtering the plurality of objects on the basis of area information, edge information, and curvature information about each of the plurality of objects, and sorting out a misrecognized target object corresponding to a preset misrecognition criterion.

Description

Image processing method and device

[0001] The present disclosure relates to a technique for selecting objects in an image.

[0002] Image analysis refers to the process of extracting meaningful information from images. It primarily involves using digital image processing techniques to extract specific objects, characteristics, and situations from an image, and then analyzing the image based on this information.

[0003] In particular, as the importance of image analysis increases in various industries, such as the advancement of image-based artificial intelligence and autonomous driving technology using cameras, various technologies for image analysis are being developed.

[0004] Although object recognition and classification within images are automated through computer processing, there are difficulties in accurately recognizing objects due to various factors when multiple objects of similar form exist.

[0005] In addition, due to the advancement of secondary battery technology, technology for studying the physical properties of materials (particles) by capturing their shapes in images using devices such as material microscopes is also developing. However, in the case of fine particles, when capturing them as two-dimensional images, it is difficult to clearly distinguish and recognize the particles due to overlapping, breakage, and foreign substances between particles.

[0006] The present embodiments aim to provide a technology for accurately selecting and recognizing multiple objects within an image.

[0007] An embodiment of the present disclosure derived to solve the aforementioned problem may provide an image processing device comprising an object identification unit that distinguishes and identifies a plurality of objects included in an image, and an object selection unit that filters a plurality of objects based on area information, border information, and curvature information for each of the plurality of objects to select misrecognition target objects corresponding to a preset misrecognition criterion.

[0008] In addition, one embodiment may provide an image processing method for object selection, comprising: a step of distinguishing and identifying a plurality of objects included in an image; and a step of filtering the plurality of objects based on area information, border information, and curvature information for each of the plurality of objects to select a misrecognition target object corresponding to a preset misrecognition criterion.

[0009] In addition, one embodiment may provide an image processing system comprising: an image processing device including an object selection unit that distinguishes and identifies a plurality of objects included in an image, and filters the plurality of objects based on area information, edge information, and curvature information for each of the plurality of objects to select a misrecognition target object corresponding to a preset misrecognition criterion; and an evaluation device that calculates at least one of a foreign matter content ratio and sphericity using information on a plurality of objects and a misrecognition target object.

[0010] According to the present embodiment, a technology for accurately selecting and recognizing multiple objects within an image can be provided.

[0011] FIG. 1 is a drawing for explaining the configuration of an image processing device according to one embodiment.

[0012] FIG. 2 is a diagram illustrating an operation for identifying a plurality of objects according to one embodiment.

[0013] FIG. 3 is a diagram illustrating an object selection operation according to one embodiment.

[0014] FIG. 4 is a diagram illustrating the operation of calculating area information according to different algorithms according to one embodiment.

[0015] FIG. 5 is a diagram illustrating an example in which first candidate objects are selected according to one embodiment.

[0016] FIG. 6 is a diagram illustrating the operation of selecting a second candidate object according to one embodiment.

[0017] FIG. 7 is a drawing for explaining an example of a first candidate object selected to a second candidate object according to one embodiment.

[0018] FIG. 8 is a diagram illustrating the operation of calculating curvature information according to one embodiment.

[0019] FIG. 9 is a diagram illustrating an example in which a misrecognized target object is selected according to one embodiment.

[0020] FIG. 10 is a flowchart illustrating an image processing method according to one embodiment.

[0021] FIG. 11 is a configuration diagram for explaining an image processing system according to one embodiment.

[0022] FIG. 12 is a diagram illustrating the operation of calculating the foreign matter content ratio according to one embodiment.

[0023] FIG. 13 is a diagram illustrating a sphericity calculation operation according to one embodiment.

[0024] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the embodiments, if it is determined that a detailed description of related known components or functions may obscure the essence of the technical concept, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless otherwise specified.

[0025] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0026] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0027] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0028] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0029] The embodiments are described in detail below with reference to the drawings.

[0030] When images are acquired using a device capable of capturing microparticles, such as a materials microscope, various physical properties can be extracted from the images. Additionally, the features of the images can be used to infer the characteristics of the microparticles.

[0031] For example, the sphericity of microparticles included in an image or the proportion of foreign substances can be verified through image analysis.

[0032] However, unlike images of a general macroscopic environment, images captured in a microscopic environment may be difficult to clearly define the boundaries of particles, and there may be difficulties in accurately calculating the features of objects within the image due to irregular shapes of fine particle clusters.

[0033] In particular, image processing technology refers to all forms of information processing regarding input data of images and videos, and most interpret images as two-dimensional signals and apply standard signal processing techniques to them. Therefore, it is difficult to distinguish overlaps or clustering between objects included within a two-dimensional image.

[0034] For example, when performing tasks such as analyzing the characteristics of individual particles or detecting foreign objects within an overall image in an image containing a large number of spherical particles, such as cells or cathode materials, particle overlap or clumping causes misidentification during analysis. Therefore, to resolve this and achieve more accurate image analysis, it is necessary to recognize and remove or separate objects corresponding to particle overlap or clumping.

[0035] With the recent advancement of deep learning technology, various zero-shot foundation vision models that take simple prompts as input, such as grounding DINO, have emerged; however, their performance degrades when images are low-quality and contain a significantly large number of objects. Consequently, if existing zero-shot deep learning technology is used, particle fragmentation / clustering cannot be recognized at all in low-magnification images containing a large number of low-quality particles.

[0036] Another method involves manually labeling particle breakage / clumping and then training a deep learning model like YOLO to recognize objects with those features; however, this method is time-consuming and has significant disadvantages in terms of cost, as it requires labeling individual objects for training. Furthermore, even when training a deep learning model through labeling, the recognition rate for clumped / overlapping particles may decrease depending on the training data.

[0037] In this situation, the present disclosure aims to provide a technology that can distinguish particle clumping / overlapping that may be misidentified within an image using only an image processing method, without separate deep learning technology. Through this, particle breakage / clumping can be effectively processed even in low-magnification image environments (where multiple particles exist within low-quality images).

[0038] Although the present disclosure describes the method based on images, it can be equally applied to image processing and analysis through video recording. For example, image processing can be performed by converting a captured video into an image based on a specific frame and applying the technology described below. Furthermore, the following description provides examples focusing on images of microparticles, but is not limited thereto. That is, the technology according to the present disclosure can also be applied to images containing various objects exhibiting similar shapes.

[0039]

[0040] FIG. 1 is a drawing for explaining the configuration of an image processing device according to one embodiment.

[0041] Referring to FIG. 1, the image processing device (100) may include an object identification unit (110) that distinguishes and identifies a plurality of objects included in an image, and an object selection unit (120) that filters a plurality of objects based on area information, border information, and curvature information for each of the plurality of objects to select a misrecognition target object corresponding to a preset misrecognition criterion.

[0042] The image processing device (100) can perform the operation of identifying multiple objects in an image when an image is input. For example, the object identification unit (110) can identify objects included in the image using a preset object recognition model. For example, the object identification unit (110) can generate a binary image that specifies objects by identifying objects present in the image using an object identification model such as the Segment-Anything model (SAM). In the binary image, the background can be set to 0 and the objects to 1. In addition, the object identification unit (110) can identify objects included in the image by separating them through various known object detection models. Through this, the object identification unit (110) can identify multiple objects included in the image by separating them from the background.

[0043] The object selection unit (120) can perform filtering on each of the identified multiple objects to select the object to be misidentified. The object to be misidentified can be determined through parameter settings, etc. For example, the object to be misidentified may refer to an object identified by overlapping or overlapping multiple objects. That is, when multiple objects overlap on a two-dimensional image, their shape may be recognized differently from other objects and thus misidentified. Therefore, in order to separate and select the object recognized as overlapping, the overlapping object can be set as the object to be misidentified.

[0044] For example, the object selection unit (120) can select a first candidate object based on area information to select a misidentified object, select a second candidate object using border information for the first candidate object, and select a misidentified object using the curvature information for the second candidate object. That is, the object selection unit (120) can finally select a misidentified object through filtering. Through filtering, a first candidate object is extracted from a plurality of objects, and a second candidate object is extracted from the first candidate object. In addition, a misidentified object is finally selected from the second candidate object.

[0045] Each filtering operation is explained below.

[0046] The object selection unit (120) can calculate area information for each object using different algorithms that are pre-set, and can select a first candidate object based on the area comparison result obtained by comparing the calculated area information. For example, the object selection unit (120) can calculate area information for individual objects identified by the object identification unit (110). The area information can be calculated using different algorithms. That is, two pieces of area information can be calculated for a single object. The object selection unit (120) can select a first candidate object using the two calculated pieces of area information.

[0047] For example, the first candidate object may be an object among a plurality of objects for which the area comparison result is greater than or equal to a preset value. For example, the object selection unit (120) may select an object as the first candidate object if the difference between the area calculated through the first algorithm and the area calculated through the second algorithm is greater than or equal to a preset value or a value set through parameter adjustment. For example, different preset algorithms may include the ConvexHull algorithm and the ConcaveHull algorithm. If the first area calculated according to the ConvexHull algorithm is greater than or equal to a preset value than the second area calculated according to the ConcaveHull algorithm, the object may be selected as the first candidate object.

[0048] As another example, the first candidate object may be an object in which the first area calculated according to the ConvexHull algorithm is larger than the second area calculated according to the ConcaveHull algorithm, and the ratio of the second area to the first area is smaller than a preset value. That is, if the value with the first area as the denominator and the second area as the numerator is less than the preset value, the object may be selected as the first candidate object. Here, the preset value may be set to a value between 0 and 1, and may be varied as a parameter value as needed.

[0049] When there are multiple similar-shaped objects within an image, normal objects (e.g., circular particles) can be excluded from the first candidate objects through a filtering operation using area information. For example, the first candidate objects may include objects such as particle breakage, particle overlap / clumping, or foreign substances of different shapes.

[0050] Meanwhile, the object selection unit (120) can select a second candidate object based on the border information of each first candidate object and pixel information specified based on the border information. For example, the border information can be set as an overlapping portion of the area outline used to calculate the area information of the first candidate object and the outline of the area difference area selected according to a different algorithm.

[0051] For example, for each of the first candidate objects, border information can be extracted. Here, the border information may be a specific section of the outline of the first candidate object. To select the first candidate object, the first area and the second area were calculated using different algorithms. The first area and the second area have an area difference. The object selection unit (120) can calculate the outline of the area difference area, which is the area of ​​difference between the first area and the second area. Additionally, the object selection unit (120) can calculate the area outline used to calculate the first area or the second area. The object selection unit (120) can calculate the section where the outline of the area difference area and the area outline overlap as border information. For example, the line of the part where the area difference outline and the outline of the area (second area) calculated through the ConcaveHull algorithm overlap can be calculated as border information.

[0052] For example, a second candidate object can be selected based on the average value of pixel information that is separated by a preset number of pixels in the normal direction of the border information. For example, the second candidate object may be an object whose average value of pixel information exceeds a set value. Here, the set value may be set to 0.

[0053] The object selection unit (120) can check pixel information for pixels separated by a certain number of pixels in the normal direction from the border information. Since the pixel information is identified in the form of a binary image by the object identification unit (110), it can have a value of 0 or 1. The object selection unit (120) calculates the average value of the pixel information and selects the object as a second candidate object if the average value appears to exceed a set value. Here, the set value can be set to 0.

[0054] Alternatively, the second candidate object may be an object in which the average value of the pixel information is greater than or equal to the parameter value set. That is, a value other than 0 may be set as the reference value. In addition, pixel information may be calculated for each pixel constituting the border, or pixel information in the normal direction may be calculated based on N pixels (where N is greater than or equal to 1 and less than the total number of pixels constituting the border) that are randomly or evenly spaced among the pixels constituting the border.

[0055] Through this secondary filtering operation, broken particles are removed, and overlapping objects or foreign objects can be selected as second candidate objects.

[0056] The object selection unit (120) can select misidentified objects by calculating multiple curvature information for a specific point among the edges of the second candidate object, calculating a ratio in which the curvature information is less than or equal to a preset value, and removing objects in which the calculated ratio is greater than a preset reference ratio.

[0057] For example, the object selection unit (120) can estimate the polygon with the highest similarity to the shape of the border and calculate curvature information using the angle between adjacent vectors with the line connecting the vertices of the polygon as a vector. The polygon with the highest similarity to the shape of the border can be selected through a pre-set polygon similarity calculation algorithm. Through this, the circular object can be represented as a polygon and can have multiple vertices.

[0058] The angle of adjacent vectors connecting each vertex can be calculated. The angle will appear as a large value when the object is circular, and as the object takes on a polygonal shape, such as a foreign substance, there will be more points showing a small value.

[0059] Accordingly, the object sorting unit (120) can calculate the ratio of an angle less than or equal to a preset value in a set of curvature information calculated using the angle between adjacent vectors. The object sorting unit (120) can determine that the object is a foreign substance if the angle less than or equal to the preset value is greater than a preset reference value.

[0060] Accordingly, the object selection unit (120) can remove the object determined to be a foreign substance from the second candidate object and select the remaining object as a misidentification target object. The misidentification target object may be an object identified as a combination of multiple objects.

[0061] Through the above operations, a learning process via labeling is not required, and only superimposed objects can be selected from images containing multiple objects with similar shapes at low magnification. These superimposed objects can be utilized in calculations such as the foreign matter content ratio or sphericity, and can serve as a more accurate image evaluation factor.

[0062] Below, the operation of each filtering step described above and various embodiments are explained in more detail with reference to the drawings.

[0063] The present disclosure relates to a technique for identifying specific objects subject to misrecognition in image information. For example, when multiple circular particles exist within an image, the clumping or overlapping of the circular particles can be distinguished and identified. Below, an operation for selecting clumped or overlapping objects (objects subject to misrecognition) in image information containing multiple circular particles, such as cathode materials, is described as an example. This is for the purpose of facilitating understanding, and the present disclosure may also be applied in cases where objects of other shapes are included.

[0064] FIG. 2 is a diagram illustrating an operation for identifying a plurality of objects according to one embodiment.

[0065] Referring to FIG. 2, the image information may include a plurality of circular objects (200). When capturing a material such as an anode material, particles of similar shape may be included in the image, and foreign substances or broken or clumped objects may appear. This is a problem associated with capturing a two-dimensional image, and it is necessary to efficiently select broken or clumped objects.

[0066] For example, an image processing device may include an object identification unit that distinguishes and identifies multiple objects contained in an image. Object identification may be performed by an algorithm that distinguishes objects from the background.

[0067] For example, objects (200) within an image can be distinguished and identified through an instance segmentation mask generated from a Segment-Anything model (SAM). A segment-anything model (SAM) is a computer vision AI foundation model that is configured to perform various tasks by pre-training on a large dataset. This model introduces the concept of "prompt-engineering" to the image domain, predicts a final mask by combining the prompt embedding and the embedding value of the original image through a prompt encoder during inference, and demonstrates good performance even with new data distributions.

[0068] In the SAM model, the prompt can be anything containing information about the target, such as a rough bounding box or a brief description of the target; if no prompt is provided, the entire image is divided into a grid, and the grid points are provided as prompts to predict the final mask for the entire image.

[0069] The object identification unit can identify all objects (200) present in the image through SAM. A binary image is generated for each object in the image, where the background is 0 and the object is 1. For convenience, the binary image for a single object can be described as a label, and the group of label values ​​for all objects in the image can be described as an object mask.

[0070] The object identification unit takes an object mask (H x W) generated from the Segment-Anything Model (SAM) as input. Here, the object mask represents an instance segmentation result image for all objects (200) existing in the original image (H x W) as shown in FIG. 2.

[0071] The object selection unit receives individual label values ​​within the object mask from SAM one by one as input and can finally extract label values ​​corresponding to overlap / clustering.

[0072] FIG. 3 is a diagram illustrating an object selection operation according to one embodiment.

[0073] Referring to FIG. 3, in the case of objects corresponding to particle overlap / clustering among multiple objects detected by the object identification unit, misrecognition and operational errors may occur, so it is necessary to perform a task to separate or remove the overlap / clustering. Detection for the separation / removal of overlap / clustering can be determined by gradually narrowing down the candidate group through a total of three stages.

[0074] For example, the object selection unit can select a first candidate object based on area information (S300). The first candidate object is selected based on area information for each label, and normal objects can be removed through the first candidate object.

[0075] Additionally, the object selection unit can select a second candidate object for the first candidate object using the boundary information (S310). The second candidate object can be selected through a Normal Vector test using pixel information of the normal vector for the boundary. The second candidate object can be selected as a set of objects from which broken particles have been removed from the first candidate object.

[0076] Subsequently, the object selection unit can select the object to be misidentified using the curvature information for the second candidate object (S320). Particles that are not determined to be circular particles through the curvature information may be classified as foreign substances, and the object remaining after the foreign substance object is removed is finally selected as the object to be misidentified. That is, the object to be misidentified refers to an object identified as two or more objects clustered together or overlapping.

[0077] The following three stages of filtering operations will be explained with reference to the drawings.

[0078] In step S310, area information is calculated for individual objects using different algorithms. Subsequently, a first candidate object can be selected using the area comparison results obtained by comparing the area information.

[0079] FIG. 4 is a diagram illustrating the operation of calculating area information according to different algorithms according to one embodiment.

[0080] Referring to FIG. 4, different algorithms may include the ConvexHull algorithm and the ConcaveHull algorithm. For a specific object, the internal area can be calculated using the ConvexHull algorithm. Additionally, for the same object, the internal area can be calculated using the ConcaveHull algorithm. The areas calculated by different algorithms for the same object may be identical or show a significant difference. If a significant difference is observed, this can be used to select a first candidate object.

[0081] For example, the following process is performed for individual objects identified within the image information. Through this, objects corresponding to overlapping / clumping / breaking particles and other foreign substances can be selected and detected.

[0082] A) Extract contour information for the label value (individual object).

[0083] For example, border information is calculated for individual objects. Border information refers to the boundary lines of areas in an image that have the same color or the same pixel value. Contour information represents list information consisting of points that constitute the contour.

[0084] B) Calculate the corresponding ConvexHull and ConcaveHull from the boundary values. For example, the ConvexHull area and ConcaveHull area are calculated based on list information consisting of points constituting the contour. Here, the algorithm for calculating the ConcaveHull area is described as the ConcaveHull algorithm. Similarly, the algorithm for calculating the ConvexHull area is described as the ConvexHull algorithm. However, this is for the sake of convenience of understanding, and any algorithm capable of calculating the ConvexHull area and ConcaveHull area for the points may correspond to different algorithms of the present disclosure, without limitation.

[0085] The Convex Hull area is calculated as 410. The Concave Hull area can be calculated as 400.

[0086] C) Assuming the internal area of ​​the Convex Hull is v and the internal area of ​​the Concave Hull is c, the first candidate object is selected based on the difference between the two areas.

[0087] For example, the first candidate object refers to an object among multiple objects for which the area comparison result is greater than or equal to a preset value. For instance, if the difference between v and c is greater than or equal to a preset value, the object may be selected as the first candidate object by determining that it does not possess a normal circular particle shape.

[0088] As another example, the first candidate object may refer to an object in which the first area (v) calculated according to the ConvexHull algorithm is larger than the second area (c) calculated according to the ConcaveHull algorithm, and the ratio of the second area to the first area is smaller than a preset value. For example, c <v이고, c / v < K 인 경우에 해당 label(객체)는 입자의 겹침 / 뭉침 후보군으로 선별될 수 있다. 여기서, K는 [0,1]사이의 np.float32값으로 사용자 설정 파라미터일 수 있다.

[0089] FIG. 5 is a diagram illustrating an example in which first candidate objects are selected according to one embodiment.

[0090] Referring to FIG. 5, overlapping or clustered particles are detected in the image information through the first candidate object selection operation described above (510). Additionally, broken particles within the image information are also detected through the first candidate object selection operation (520). Likewise, foreign substances are also detected through the first candidate object selection operation (530).

[0091] In this way, by excluding circular normal particles using area information, the object selection unit can select various candidate objects, including objects subject to misidentification. Since the first candidate object may include not only objects subject to misidentification but also other objects such as foreign substances and broken objects, an additional filtering operation is required.

[0092] FIG. 6 is a diagram illustrating the operation of selecting a second candidate object according to one embodiment.

[0093] The object selection unit can select a second candidate object based on the boundary information of each first candidate object and pixel information specified based on the boundary information. For example, the boundary information may be set as an overlapping portion of the area outline used to calculate the area information of the first candidate object and the outline of the area difference area selected according to a different algorithm.

[0094] Referring to FIG. 6, when calculating the area of ​​an object using different algorithms used in selecting a first candidate object, an area difference area (620) may occur. The area difference area is defined as the area (620) where there is a difference between the ConvexHull area and the ConcaveHull area. Additionally, the outline used when calculating the ConcaveHull area and the outline of the 620 area partially overlap (610). The border information (610) is set to the overlapping outline.

[0095] The object selection unit obtains pixel information for pixels (630) separated by a preset number of pixels in the normal direction of the overlapping outline, which is the border information (610). The pixel information may have a value of 0 or 1, or a value between 0 and 1. Once information for each pixel is obtained, the average value of the obtained pixel information for the object is calculated. The object selection unit determines whether the object is selected as a second candidate object using the average value of the pixel information. For example, an object whose average value exceeds a set value may be selected as a second candidate object. The spacing of the separated pixels (630) can be set according to parameters. For example, the distance from the border of 630 pixels can be dynamically determined based on the density of objects within the image, etc. In an environment where objects within the image are dense, there is a high possibility that the pixel (630) value will be affected by other objects. Therefore, when the density of objects within the image is high, the distance may be set to be short. Conversely, when the object density within the image is low, the separation distance may be set to be long.

[0096] This filtering process can be named the Normal Vector test.

[0097] For example, as shown in Fig. 6, a normal vector in the direction outward of the object can be calculated in the border area where there is a difference in area between the Convex Hull and the Concave Hull (the border area of ​​the primary candidate object selected from the primary screening). The object screening unit examines pixel values ​​in the direction of the normal vector, and if the average value of the corresponding pixels exceeds a set value (e.g., a background such as 0), it performs secondary screening as a particle / overlap candidate group. Through the Normal Vector test, particle breakage can be excluded from the primary candidate objects obtained from the primary screening, and the secondary candidate objects that undergo the test may include particle overlap / clumping and other foreign substances.

[0098] FIG. 7 is a drawing for explaining an example of a first candidate object selected to a second candidate object according to one embodiment.

[0099] Referring to FIG. 7, 710 represents the image before the aforementioned second candidate object selection. 720 represents the image after the second candidate object selection. In 710, it can be seen that broken objects (711, 712) were selected along with nested objects. Subsequently, in image 720 after the Normal Vector test, it can be seen that objects 711 and 712 were excluded from selection.

[0100] In this way, particle breakage can be excluded from primary candidate objects obtained from primary screening through the Normal Vector test, and secondary candidate objects that have undergone the test may contain one of particle overlap / clumping and other foreign substances.

[0101] Meanwhile, the object selection unit can select misrecognition target objects by calculating multiple curvature information for a specific point among the boundaries of the second candidate object, calculating a ratio in which the curvature information is less than or equal to a preset value, and removing objects in which the ratio is greater than a preset reference ratio.

[0102] For example, the object selection unit can estimate the polygon with the highest similarity to the shape of the border, and calculate curvature information using the angle between adjacent vectors with the line connecting the vertices of the polygon as a vector.

[0103] FIG. 8 is a diagram illustrating the operation of calculating curvature information according to one embodiment.

[0104] Referring to Fig. 8, misidentified target objects can be classified and selected by performing a Curvature test using curvature information.

[0105] For example, a test can be performed to remove values ​​corresponding to foreign substances from the second group of candidate objects selected from the second screening. Since particle overlap / clumping originates from circular objects, they possess a certain degree of circular curvature. Therefore, the curvature of each object is measured through the following logic, and if the ratio of measured values ​​smaller than L (a user-defined parameter of np.float32) is greater than or equal to H (a user-defined parameter of np.float32, a reference ratio) (i.e., if it possesses curvature smaller than a circle), the corresponding label (object) can be determined to be a foreign substance and removed. Labels (objects) that pass the curvature test are ultimately objects corresponding to particle overlap / clumping (objects subject to misidentification), thereby enabling effective detection of particle overlap / clumping phenomena.

[0106] Specifically, the identification of objects subject to misrecognition can be performed through the following actions.

[0107] A) Extract border information (Contour) for Label values

[0108] For example, the object selection unit can extract border information for each of the second candidate objects.

[0109] B) Estimation of the polygon most similar to the extracted boundary values, followed by set calculation for the vertices forming the polygon

[0110] For example, the object selection unit can estimate the polygon most similar to the boundary value of each second candidate object and calculate the vertices (P2) that form the polygon. Each polygon can be pre-set, and as shown in FIG. 8, the polygon most similar to the shape of each second candidate object can be estimated according to a pre-set algorithm. Each polygon may include multiple vertices that form the polygon.

[0111] C) For all vertex values ​​included in the set, assuming the current point is P1, and the next points are P2 and P3 in order, the vector connecting P1 and P2 is , the vector connecting P2 and P3 It can be specified as such. Afterwards, the angle between the two vectors can be calculated using the inner and outer products of v1 and v2 as shown in Equation 1 below.

[0112] [Formula 1]

[0113] Angle = atan2(cross product, dot product)

[0114] For example, a vector connecting adjacent vertices (P1 and P3) can be calculated based on each vertex (P2), and the angle (theta) formed by that vector can be calculated. Depending on the polygon, there may be multiple vertices, and the angle at each vertex can be calculated. That is, for each vertex, the angle formed by v1 and v2 is calculated, and if the proportion of such angles that appear smaller than a preset L is greater than or equal to H, the object can be determined to be a foreign substance. In other words, the object determined to be a foreign substance contains a certain proportion of vertices with small angles, which differs from the shape of a circular object, and is therefore classified as a foreign substance object.

[0115] Here, L and H can be set dynamically and can also be set as parameters determined by user input.

[0116] The object selection unit can finally select the object to be misidentified by removing foreign objects from the second group of candidate objects through the Curvature test in this way.

[0117] FIG. 9 is a diagram illustrating an example in which a misrecognized target object is selected according to one embodiment.

[0118] Referring to FIG. 9, objects captured in overlap within image information can be distinguished and selected through the aforementioned three-step filtering. For example, objects identified as overlapping beneath specific objects, such as 910 to 940, can be distinguished and selected as overlapping objects. This allows for more accurate identification by enabling the distinction and identification of objects that are not identified in a circular form when overlapping with the SAM image to which the operation according to the present disclosure is applied.

[0119] The misidentified objects selected through this operation can help calculate more accurate values ​​in the foreign matter content ratio and sphericity calculations described below.

[0120] As such, when circular particles overlap or clump together within an image, they can be detected and, if necessary, removed or clustered for differentiation. In particular, when the industrial sector attempts to observe circular particles (e.g., cathode materials, cells, etc.) in low-magnification SEM images, it is nearly impossible for an operator to manually arrange countless small particles in the micro / nanometer range without overlapping during the SEM capture. Since particle overlap or clumping within an image can lead to analysis errors during subsequent analysis stages, a technology capable of detecting this phenomenon is required.

[0121] Under these circumstances, the aforementioned embodiments can effectively detect particle overlap / clustering phenomena without separate deep learning technology. Therefore, there is an advantage in that a GPU is not used in the detection step and data labeling for deep learning training is not required.

[0122] The operation of the aforementioned image processing device is briefly explained once again below. The operation of the aforementioned image processing device can be performed in each step described below. Furthermore, the steps described below are distinguished for ease of understanding, and the integration or division of steps also falls within the scope of this embodiment.

[0123] FIG. 10 is a flowchart illustrating an image processing method according to one embodiment.

[0124] Referring to FIG. 10, the image processing method may include a step of distinguishing and identifying a plurality of objects included in the image (S1000).

[0125] The step of distinguishing and identifying objects can perform the operation of identifying multiple objects in an image when an image is input. For example, the object identification step (S1000) can identify objects included in the image using a pre-configured object recognition model. For example, the object identification step (S1000) can generate a binary image that specifies objects by identifying objects existing in the image using an object identification model such as the Segment-Anything model (SAM). In the binary image, the background can be set to 0 and the objects to 1. In addition, the object identification step (S1000) can separate and identify objects included in the image through various known object detection models. Through this, the object identification step (S1000) can identify multiple objects included in the image by separating them from the background.

[0126] The image processing method may include a step of filtering multiple objects based on area information, border information, and curvature information for each of the multiple objects to select a misrecognition target object corresponding to a preset misrecognition criterion (S1010).

[0127] The object selection step (S1010) can select objects subject to misrecognition by performing filtering on each of the identified multiple objects. Objects subject to misrecognition can be determined through parameter settings, etc. For example, objects subject to misrecognition may refer to objects identified as overlapping or overlapping multiple objects. That is, when multiple objects overlap on a 2D image, their shape may be recognized differently from other objects and thus misrecognized. Therefore, in order to separate and select objects recognized as overlapping, overlapping objects can be set as objects subject to misrecognition.

[0128] For example, the object selection step (S1010) can select a first candidate object based on area information to select a misidentified object, select a second candidate object using boundary information for the first candidate object, and select a misidentified object using the curvature information for the second candidate object. That is, the object selection step (S1010) can finally select a misidentified object through sequential filtering. Through sequential filtering, a first candidate object is extracted from multiple objects, and a second candidate object is extracted from the first candidate object. Furthermore, a misidentified object is finally selected from the second candidate object.

[0129] Specifically, the object selection step (S1010) can calculate area information for each object using different algorithms that are pre-set, and select a first candidate object based on the area comparison result obtained by comparing the calculated area information. For example, the object selection step (S1010) can calculate area information for individual objects identified in the object identification step (S1000). The area information can be calculated using different algorithms. That is, two pieces of area information can be calculated for a single object. The object selection step (S1010) can select a first candidate object using the two calculated pieces of area information.

[0130] For example, the first candidate object may be an object among multiple objects for which the area comparison result is greater than or equal to a preset value. For example, the object selection step (S1010) may select an object as the first candidate object if the difference between the area calculated through the first algorithm and the area calculated through the second algorithm is greater than or equal to a preset value or a value set through parameter adjustment. For example, the different preset algorithms may be the ConvexHull algorithm and the ConcaveHull algorithm. If the first area calculated according to the ConvexHull algorithm is greater than or equal to a preset value than the second area calculated according to the ConcaveHull algorithm, the object may be selected as the first candidate object.

[0131] As another example, the first candidate object may be an object in which the first area calculated according to the ConvexHull algorithm is larger than the second area calculated according to the ConcaveHull algorithm, and the ratio of the second area to the first area is smaller than a preset value. That is, if the value with the first area as the denominator and the second area as the numerator is less than the preset value, the object may be selected as the first candidate object. Here, the preset value may be set to a value between 0 and 1, and may be varied as a parameter value as needed.

[0132] Meanwhile, the object selection step (S1010) can select a second candidate object based on the boundary information of each first candidate object and pixel information specified based on the boundary information. For example, the boundary information may be set as an overlapping portion of the area outline used to calculate the area information of the first candidate object and the outline of the area difference area selected according to a different algorithm.

[0133] For example, for each first candidate object, border information can be extracted. Here, the border information may be a specific section of the outline of the first candidate object. To select the first candidate object, the first area and the second area were calculated using different algorithms. The first area and the second area have an area difference. The object selection step (S1010) can calculate the outline of the area difference area, which is the area of ​​difference between the first area and the second area. Additionally, the object selection step (S1010) can calculate the area outline used to calculate the first area or the second area. The object selection step (S1010) can calculate the section where the outline of the area difference area and the area outline overlap as border information. For example, the line of the part where the area difference outline and the outline of the area (second area) calculated through the ConcaveHull algorithm overlap can be calculated as border information.

[0134] For example, a second candidate object can be selected based on the average value of pixel information that is separated by a preset number of pixels in the normal direction of the border information. For example, the second candidate object may be an object whose average value of pixel information exceeds a set value. Here, the set value may be set to 0.

[0135] The object selection step (S1010) can check pixel information for pixels separated by a certain number of pixels in the normal direction from the border information. Since the pixel information is identified in the form of a binary image in the object identification step, it can have a value of 0 or 1. The object selection step (S1010) calculates the average value of the pixel information and selects the object as a second candidate object if the average value appears to exceed a set value. Here, the set value can be set to 0.

[0136] Alternatively, the second candidate object may be an object in which the average value of pixel information is greater than or equal to the parameter value set. That is, a value other than 0 may be set as the reference value. Additionally, pixel information may be calculated for each pixel constituting the border, or pixel information in the normal direction may be calculated based on N pixels (where N is greater than or equal to 1 and less than the total number of pixels constituting the border) that are randomly or evenly spaced among the pixels constituting the border. Through this secondary filtering operation, broken particles may be removed, and overlapping objects or foreign matter objects may be selected as the second candidate objects.

[0137] Finally, the object selection step (S1010) can select misrecognition target objects by calculating multiple curvature information for a specific point among the boundaries of the second candidate object, calculating a ratio in which the curvature information is less than or equal to a preset value, and removing objects in which the calculated ratio is greater than a preset reference ratio.

[0138] For example, the object selection step (S1010) can estimate the polygon with the highest similarity to the shape of the border and calculate curvature information using the angle between adjacent vectors with the line connecting the vertices of the polygon as a vector. The polygon with the highest similarity to the shape of the border can be selected through a pre-set polygon similarity calculation algorithm. Through this, the circular object can be represented as a polygon and can have multiple vertices.

[0139] The angle of adjacent vectors connecting each vertex can be calculated. The angle will appear as a large value when the object is circular, and as the object takes on a polygonal shape, such as a foreign substance, there will be more points showing a small value.

[0140] Accordingly, the object selection step (S1010) can calculate the ratio of angles less than or equal to a preset value in a set of curvature information calculated using the angle between adjacent vectors. The object selection step (S1010) can determine that the object is a foreign substance if the angle less than or equal to the preset value is greater than a preset reference value.

[0141] Accordingly, the object selection step (S1010) can remove the object determined to be a foreign substance from the second candidate object and select the remaining object as the object to be misidentified. The object to be misidentified may be an object identified as a combination of multiple objects.

[0142] Through the above operations, a learning process via labeling is not required, and only superimposed objects can be selected from images containing multiple objects with similar shapes at low magnification. These superimposed objects can be utilized in calculations such as the foreign matter content ratio or sphericity, and can serve as a more accurate image evaluation factor.

[0143] Below, a system for evaluating an image using the case where the misrecognized target object described with reference to FIGS. 1 to 10 is distinguished and selected is described.

[0144] FIG. 11 is a configuration diagram for explaining an image processing system according to one embodiment.

[0145] Referring to FIG. 11, the image processing system includes an image processing device (100) that distinguishes and identifies multiple objects included in an image, and filters multiple objects based on area information, border information, and curvature information for each of the multiple objects to select a misrecognition target object corresponding to a preset misrecognition criterion, and an evaluation device (1100) that calculates at least one of a foreign matter content ratio and sphericity using information on multiple objects and a misrecognition target object.

[0146] The evaluation device (1100) may refer to a device that analyzes an image to extract various information. For example, the evaluation device (1100) may calculate the foreign matter content ratio, which is the ratio of foreign matter content in an object included in an image, through image analysis. Alternatively, the evaluation device (1100) may evaluate the degree of sphericity by calculating a sphericity index regarding how spherical an object included in an image is.

[0147] In addition to this, the evaluation device (1100) can produce various evaluation factors through image analysis, and there are no limitations thereon.

[0148] In the case of the image processing device (100), the operation described with reference to FIGS. 1 to 10 may be performed. To avoid redundant descriptions, the specific operation of the image processing device (100) is omitted.

[0149] Analysis techniques utilizing image information are being used in various industrial fields. For example, this system can be used to capture cell images and analyze whether abnormal cells exist and to determine their proportion. Alternatively, for materials such as secondary batteries, evaluation through image analysis can be performed to verify product quality during the material manufacturing process and to filter out defective products. In this case, it is necessary to evaluate the content of the aforementioned foreign substances and the degree of sphericity of individual material particles, which should appear as a circle.

[0150] However, in this case, as described above, if the objects overlap, they may be misjudged as not being spherical, or may be misidentified as foreign substances, etc. Therefore, the image processing system can increase the evaluation reliability of the evaluation device (1100) by utilizing the misidentified objects obtained from the image processing device (100).

[0151] First, the operation of calculating the foreign matter content ratio of the evaluation device (1100) will be explained.

[0152] The model for calculating the foreign matter content ratio can automatically detect foreign matter and calculate the ratio on low-magnification SEM images without manual data labeling. This model can detect particle breakage, fine particles, foreign matter, and lithium in the images. The results of the foreign matter content ratio calculation are then graded into six levels using a Gaussian Mixture Model (GMM), and levels 5 and above can be set as the upper limit for management specifications. The model is largely composed of an AI network and a rule-based model, and can be configured to facilitate easy modification of the result grading section and immediate application when defect criteria change.

[0153] Specifically, the operation of each step is explained below.

[0154] FIG. 12 is a diagram illustrating the operation of calculating the foreign matter content ratio according to one embodiment.

[0155] Referring to FIG. 12, the evaluation device can exclude a misidentified target object from a plurality of objects (S1200).

[0156] The evaluation device can remove objects set as misidentification targets among multiple objects included in an image. Here, removal refers to a masking operation.

[0157] For example, the evaluation device can identify objects in an image and apply a mask for the misidentified target object produced by the image processing device to the generated output mask to produce an instance segmentation mask (Group A) from which the misidentified target object has been removed. The mask (Group B) corresponding to the misidentified target object can be separated.

[0158] Through this, potential misidentification targets within the image information are removed first, and the analysis process can then proceed.

[0159] The evaluation device can extract foreign objects using a clustering unsupervised learning model (S1210).

[0160] For example, the evaluation device can perform patch embedding on Group A from which misidentified objects have been removed, and then perform an unsupervised learning clustering technique based on feature values ​​such as shape and size to form clusters for normal objects (e.g., active materials (circular objects)) and foreign substances (particle breakage + fine particles + foreign substances + lithium, etc.).

[0161] Since the evaluation device performed clustering only for Group A using an unsupervised learning model, it is possible to prevent cases where normal objects (target objects) are clustered as foreign objects due to the overlap of normal objects.

[0162] The evaluation device can calculate the foreign matter content ratio by calculating the foreign matter object area relative to the area of ​​a plurality of objects (S1220).

[0163] For example, when foreign matter and normal objects are distinguished and clustered through step S1210, the evaluation device calculates the area occupied by objects clustered as foreign matter within the image. Depending on the settings, the evaluation device may calculate not only the area but also the number, etc. This may vary depending on whether the foreign matter content ratio is based on area, the number, or other setting factors. The present disclosure may be applied without limitation. For convenience, the following description is based on area.

[0164] When the total area of ​​objects clustered as foreign matter is calculated, the evaluation device calculates the area of ​​all objects included in the initial image information. The evaluation device calculates the foreign matter content ratio by calculating the ratio of the foreign matter area to the total object area.

[0165] Next, the operation of the evaluation device to calculate sphericity is explained.

[0166] The degree of distortion of spherical particles can be automatically quantified on low-magnification SEM images without manual data labeling. Similar to the foreign matter content ratio calculation model, it consists of an AI network and a rule-based model, and the AI ​​model remains the same for each different process, allowing it to be applied to various processes by performing only minor fine-tuning on the rule-based model.

[0167] FIG. 13 is a diagram illustrating a sphericity calculation operation according to one embodiment.

[0168] Referring to FIG. 13, the evaluation device can exclude foreign objects from a plurality of objects (S1300).

[0169] For example, the evaluation device can preprocess the input image (Histogram equalization, Normalization, etc.) and then receive zero-shot instance segmentation results for all objects present in the image through a SAM model.

[0170] In addition, the evaluation device can remove foreign objects from multiple objects in the image by removing the mask for foreign objects extracted from the foreign object content ratio calculation of FIG. 12.

[0171] The evaluation device can select a target object by adding a misrecognition target object (S1310).

[0172] For example, a target object for sphericity calculation can be selected by adding a target object for misidentification to a plurality of objects from which foreign matter has been removed. That is, the evaluation device can detect only the object corresponding to the anode material (circular object) by using the target object for misidentification from the image processing device and the foreign object extracted from the foreign matter content ratio calculation for the output mask of the plurality of received objects.

[0173] Afterwards, the evaluation device can derive multiple spherical indicators for a target object using multiple algorithms that are pre-set (S1320).

[0174] For example, the evaluation device can derive a sphericity indicator for a target object using pre-set algorithms. As an example, the evaluation device can extract a patch corresponding to the size of each target object and use the DirectLeastSquare algorithm to detect an ideal ellipse that fits the target object. As another example, the evaluation device can calculate a sphericity indicator by calculating at least one of the Intersection Over Union (IOU), Hausdorff Distance (HD), and Aspect ratio (b / a) for the target object.

[0175] The evaluation device calculates the degree of sphericity using a sphericity indicator (S1330).

[0176] For example, the evaluation device can integrate spherical indicators to ultimately quantify the sphericity of the target object.

[0177]

[0178] The values ​​of each spheric metric can be expressed numerically, and the final sphericity can be calculated arithmetically by integrating the values ​​or applying preset weights. Through this process, the sphericity of individual target objects can be calculated.

[0179] When the sphericity of individual target objects is calculated, the evaluation device can calculate the ideal particle ratio in the entire target object and classify and provide the sphericity into grades or stages according to the ideal particle ratio.

[0180] In the case of evaluation, the device may also notify of defects by generating an alert if the sphericity drops below a specific level or grade.

[0181] In this way, the image processing system can receive a two-dimensional image and provide accurate analysis information by selecting, distinguishing, and analyzing multiple objects within the image using an image processing device and an evaluation device.

[0182] As described above, the present embodiment provides the effect of increasing analysis accuracy by accurately distinguishing overlapping particles in an image containing many similar shapes, and enabling rapid processing of image data at low cost.

[0183] The foregoing description is merely an illustrative explanation of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations within the scope of the essential characteristics of the technical concept. Furthermore, since these embodiments are intended to explain, not limit, the scope of the technical concept is not limited by these embodiments. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present disclosure.

[0184]

[0185] CROSS-REFERENCE TO RELATED APPLICATION

[0186] This patent application claims priority pursuant to Section 119(a) of the U.S. Patent Act (35 USC § 119(a)) to Korean Patent Application No. 10-2024-0149236 filed on October 29, 2024, all of which are incorporated by reference into this patent application. Additionally, this patent application claims priority in countries other than the United States for the same reasons as above, all of which are incorporated by reference into this patent application.

Claims

1. An object identification unit that distinguishes and identifies multiple objects included in an image; and An image processing device comprising an object selection unit that filters the plurality of objects based on area information, border information, and curvature information for each of the plurality of objects and selects misrecognition target objects corresponding to preset misrecognition criteria.

2. In Paragraph 1, The above object selection unit is, An image processing device that selects a first candidate object based on the above area information, selects a second candidate object using the above boundary information for the first candidate object, and selects a misrecognition target object using the above curvature information for the second candidate object.

3. In Paragraph 2, The above object selection unit is, An image processing device characterized by calculating area information for each of the above objects through different algorithms set in advance, and selecting the first candidate object based on an area comparison result obtained by comparing the calculated area information.

4. In Paragraph 3, The above-mentioned first candidate object is, An image processing device characterized by being an object among the plurality of objects above in which the area comparison result is calculated to be greater than or equal to a preset value.

5. In Paragraph 3, The above-mentioned preset different algorithms are, An image processing device characterized by including a ConvexHull algorithm and a ConcaveHull algorithm.

6. In Paragraph 5, The above-mentioned first candidate object is, An image processing device characterized by an object in which a first area calculated according to the above ConvexHull algorithm is larger than a second area calculated according to the above ConcaveHull algorithm, and the ratio of the second area to the first area is smaller than a preset value.

7. In Paragraph 2, The above object selection unit is, An image processing device characterized by selecting the second candidate object based on the border information of each of the first candidate objects and pixel information specified based on the border information.

8. In Paragraph 7, The above border information is, An image processing device characterized by the area outline used to calculate the area information of the first candidate object and the outline of the area difference area selected according to a different algorithm being set as an overlapping part.

9. In Paragraph 7, The above second candidate object is, An image processing device characterized by being selected based on the average value of the pixel information specified by being spaced apart by a preset number of pixels in the normal direction of the above-mentioned border information.

10. In Paragraph 9, The above second candidate object is, An image processing device characterized by the fact that the average value of the pixel information above is an object that exceeds a set value.

11. In Paragraph 2, The above object selection unit is, An image processing device characterized by calculating a plurality of curvature information for a specific point among the boundaries of a second candidate object, calculating a ratio in which the curvature information is less than or equal to a preset value, and removing an object in which the ratio is greater than a preset reference ratio to select the object to be misrecognized.

12. In Paragraph 11, The above object selection unit is, An image processing device characterized by estimating the polygon with the highest similarity to the shape of the above border, and calculating the curvature information using the angle between adjacent vectors with the line connecting the vertices of the above polygon as a vector.

13. In Paragraph 1, The above misrecognized object is, An image processing device characterized by the fact that the above-mentioned plurality of objects are objects identified by overlapping.

14. In an image processing method for object selection, A step of distinguishing and identifying multiple objects included in an image; and An image processing method comprising the step of filtering the plurality of objects based on area information, border information, and curvature information for each of the plurality of objects to select a misrecognition target object corresponding to a preset misrecognition criterion.

15. In Paragraph 14, The step of selecting the above-mentioned object to be misrecognized is: An image processing method that selects a first candidate object based on the above area information, selects a second candidate object using the above boundary information for the first candidate object, and selects a misrecognition target object using the above curvature information for the second candidate object.

16. In Paragraph 15, The step of selecting the above-mentioned object to be misrecognized is: An image processing method characterized by calculating area information for each of the above objects through different algorithms set in advance, and selecting the first candidate object based on an area comparison result obtained by comparing the calculated area information.

17. In Paragraph 15, The step of selecting the above-mentioned object to be misrecognized is: An image processing method characterized by selecting the second candidate object based on the border information of each of the first candidate objects and pixel information specified based on the border information.

18. In Paragraph 15, The step of selecting the above-mentioned object to be misrecognized is: An image processing method characterized by calculating a plurality of curvature information for a specific point among the boundaries of a second candidate object, calculating a ratio in which the curvature information is less than or equal to a preset value, and removing an object in which the ratio is greater than a preset reference ratio to select the object to be misrecognized.

19. In an image processing system, An image processing device comprising an object selection unit that distinguishes and identifies a plurality of objects included in an image, and filters the plurality of objects based on area information, border information, and curvature information for each of the plurality of objects to select misrecognition target objects corresponding to preset misrecognition criteria; and An image processing system comprising an evaluation device that calculates at least one of a foreign matter content ratio and sphericity using the above-mentioned plurality of objects and the above-mentioned misrecognition target object information.

20. In Paragraph 19, The above evaluation device is, An image processing system characterized by excluding the object to be misrecognized from the plurality of objects, extracting the foreign object using a clustering unsupervised learning model, and calculating the foreign object content ratio by calculating the area of ​​the foreign object relative to the area of ​​the plurality of objects.

21. In Paragraph 20, The above evaluation device is, An image processing system characterized by excluding the foreign object from the plurality of objects and adding the object to be misrecognized to select a target object, and calculating the sphericity by deriving a plurality of sphericity indicators using a plurality of algorithms pre-set for the target object.

Images