Apparatus and method for video switching with multiple object

Abstract

An image matching device and method including multiple objects are disclosed. The image matching device including multiple objects according to the present invention is characterized by comprising: an overlapping area calculation module that extracts feature points from two images, groups the feature points into a triangular shape to generate a matched triangle, compares the matched triangles to calculate a similar area between the two images based on the comparison result, and then generates an overlapping area image based on the coordinate values ​​of the similar area; an error matrix extraction unit that generates an error matrix from the overlapping area image; an object extraction unit that detects the movement of multiple objects in the overlapping area image and generates a virtual boundary line that includes all detected multiple objects; a weight generation unit that generates a weight based on a minimum error boundary, wherein the value increases as the difference along the x-axis from the boundary line value of the previous image detected in the overlapping area image increases; and an image blending unit that calculates a Minimum Error Seam (MES) using the error matrix, the boundary line, and the minimum error boundary-based weight, and then matches and blends the two images based on the minimum error boundary MES.

Description

Technology Field

[0001] The present invention relates to an image registration device and method including multiple objects, and more specifically, to an image registration device and method including multiple objects that prevents damage to the time invariance of moving multiple objects when registering images captured in a horizontal direction. Background Technology

[0002] With the advancement of digital cameras, anyone can easily acquire high-quality video, and high-resolution personal devices such as smartphones and tablet PCs, which allow for effective consumption, are also being used in daily life. In conjunction with these advancements, there is an increasing desire to consume video content that matches individual consumers' video consumption purposes and the devices they use. Among these, panoramic video based on large screens is gaining interest as screen sizes increase, as it provides users with a wide field of view.

[0003] In order to produce such a panoramic image, it is essential to combine multiple images into a single image, a process called image stitching.

[0004] Image stitching technology enables the generation of panoramic images free from visual errors by sequentially utilizing algorithms based on image synthesis technology. Stitching technology has evolved with significant improvements in accuracy. However, this improvement in accuracy increases the computational load required for image processing, which can lead to a decrease in processing speed. This decrease in processing speed may limit the scope of application for panoramic image generation technology.

[0005] The background technology of the present invention is disclosed in the ‘image matching method’ of Korean Published Patent Application No. 10-2016-0047846 (May 3, 2016). The problem to be solved

[0006] The present invention was devised to improve upon the aforementioned problems, and an objective according to one aspect of the present invention is to provide an image registration device and method including multiple objects that prevent damage to the time invariance of moving multiple objects during image registration captured in a horizontal direction. means of solving the problem

[0007] A multi-object image matching device according to one aspect of the present invention is characterized by comprising: an overlapping area calculation module that extracts feature points from two images, groups the feature points into a triangular shape to generate a matched triangle, compares the matched triangle to calculate a similar area between the two images based on the comparison result, and then generates an overlapping area image based on the coordinate values ​​of the similar area; an error matrix extraction unit that generates an error matrix from the overlapping area image; an object extraction unit that detects the movement of multiple objects in the overlapping area image and generates a virtual boundary line that includes all the detected multiple objects; a weight generation unit that generates a minimum error boundary-based weight in which the value increases as the difference along the x-axis from the boundary line value of the previous image detected in the overlapping area image increases; and an image blending unit that calculates a minimum error boundary MES using the error matrix, the boundary line, and the minimum error boundary-based weight, and then matches the two images based on the minimum error boundary MES and performs blending processing.

[0008] The overlapping region calculation module of the present invention comprises: a feature point extraction unit for extracting the feature points; a feature point grouping unit for grouping each of the feature points into triangles to generate the matched triangles; a matched pair comparison unit for comparing the matched triangle pairs with each other and matching the matched triangle pairs determined to exist in the same location; a similar region setting unit for setting a minimum-sized rectangle containing all the matched triangle pairs as the similar region; and an overlapping region generation unit for generating the overlapping region image using the coordinate values ​​of the similar region.

[0009] The number of feature points of the present invention is characterized by being proportional to the size of the image.

[0010] The feature point extraction unit of the present invention extracts the feature points according to priority, and is characterized by excluding any one of the feature points depending on whether the distance between the feature points is within a preset distance.

[0011] The feature point grouping unit of the present invention is characterized by grouping three feature points located at the relatively closest positions among the feature points to generate the matched triangle.

[0012] The matching pair comparison unit of the present invention is characterized by performing the process of comparing matching pair triangles using triangle similarity conditions.

[0013] The matching pair comparison unit of the present invention is characterized by determining whether the matching triangles are similar by comparing the length ratios of each side of the matching triangle pairs.

[0014] The similar area setting unit of the present invention is characterized by setting a rectangle of the minimum size that includes all of the matched triangle pairs as the similar area.

[0015] The similar region setting unit of the present invention is characterized by setting the similar region using the maximum or minimum value among the coordinates included in the matched triangle pair.

[0016] The object extraction unit of the present invention is characterized by connecting a first line connecting the topmost pixel and the leftmost pixel among the edge pixels in the overlapping area image and a second line connecting the bottommost pixel and the leftmost pixel, finding the pixel that is relatively farthest from each of the first line and the second line, moving the first line and the second line horizontally along the Y-axis, and then connecting the leftmost pixels of the first line and the second line respectively to generate the boundary line that includes all of the multiple objects.

[0017] The image blending unit of the present invention is characterized by generating an error value through the error matrix, the boundary line, and the minimum error boundary-based weight, and generating the minimum error boundary by summing the lowest value among the top three matrix values ​​in the y-axis direction of the error value.

[0018] A method for aligning images including multiple objects according to one aspect of the present invention comprises the steps of: an overlapping region calculation module extracting feature points from two images and grouping the feature points into a triangular shape to generate aligned triangles, comparing the aligned triangles to calculate similar regions of the two images based on the comparison result, and then generating an overlapping region image based on the coordinate values ​​of the similar regions; an error matrix extraction unit generating an error matrix from the overlapping region image; an object extraction unit detecting the movement of multiple objects in the overlapping region image and generating a virtual boundary line that includes all detected multiple objects; a weight generation unit generating a minimum error boundary-based weight in which the value increases as the difference along the x-axis from the boundary line value of the previous image detected in the overlapping region image increases; and an image blending unit calculating a minimum error boundary MES using the error matrix, the boundary line, and the minimum error boundary-based weight, and then aligning the two images based on the minimum error boundary MES and performing blending processing.

[0019] The step of generating the overlapping region image of the present invention comprises: a step of extracting the feature points; a step of generating the matched triangles by grouping each of the feature points into triangles; a step of matching the matched triangle pairs determined to exist in the same location by comparing the matched triangle pairs with each other; a step of setting a minimum-sized rectangle containing all the matched triangle pairs as the similar region; and a step of generating the overlapping region image using the coordinate values ​​of the similar region.

[0020] The number of feature points of the present invention is characterized by being proportional to the size of the image.

[0021] The step of generating the matched triangle by grouping each of the feature points of the present invention into triangles is characterized by extracting the feature points according to priority, and excluding any one of the feature points depending on whether the distance between the feature points is within a preset distance.

[0022] The step of generating the matched triangle by grouping each of the feature points of the present invention into a triangle is characterized by generating the matched triangle by grouping the three feature points that are relatively closest among the feature points.

[0023] The step of matching pairs of aligned triangles determined to exist in the same location by comparing the pairs of aligned triangles of the present invention is characterized by performing the process of comparing pairs of aligned triangles using triangle similarity conditions.

[0024] The step of matching pairs of matched triangles determined to exist in the same location by comparing the matched pairs of triangles of the present invention is characterized by determining whether the matched triangles are similar by comparing the ratio of the lengths of each side of the matched pairs of triangles.

[0025] The step of setting the minimum size rectangle containing all of the matched triangle pairs of the present invention as the similar area is characterized by setting the minimum size rectangle containing all of the matched triangle pairs as the similar area.

[0026] The step of setting the similar region as a rectangle of the minimum size that includes all of the matched triangle pairs of the present invention is characterized by setting the similar region using the maximum or minimum value among the coordinates included in the matched triangle pairs.

[0027] The step of generating the virtual boundary line of the present invention is characterized by connecting a first line connecting the topmost pixel and the leftmost pixel among the edge pixels in the overlapping area image and a second line connecting the bottommost pixel and the leftmost pixel, finding the pixel at the relatively farthest distance from each of the first line and the second line, moving the first line and the second line horizontally along the Y-axis, and then connecting the leftmost pixels of the first line and the second line, respectively, to generate the boundary line that includes all of the multiple objects.

[0028] The step of aligning and blending two images based on the minimum error boundary MES of the present invention is characterized by generating an error value through the error matrix, the boundary line, and the minimum error boundary-based weight, and generating the minimum error boundary by summing the lowest value among the top three matrix values ​​in the y-axis direction of the error value. Effects of the invention

[0029] An image matching device and method including multiple objects according to one aspect of the present invention prevents damage to the time invariance of moving multiple objects when matching images captured in a horizontal direction and enables a reduction in image matching speed.

[0030] An image matching device and method including multiple objects according to another aspect of the present invention can determine similar regions with high accuracy by limiting the extraction of adjacent feature points when extracting feature points, and can eliminate repeatability in determining similar regions by processing threshold values ​​for determining similar regions in parallel.

[0031] An image matching device and method including multiple objects according to another aspect of the present invention can increase consistency by acquiring a motion image through the difference between consecutive frames of an image and transmitting it in the form of a weight to accurately detect the boundary lines of multiple objects.

[0032] An image matching device and method including multiple objects according to another aspect of the present invention can provide a more suitable stitched image in a surveillance environment where various objects appear frequently and rapid image delivery to an image observer is required, by protecting multiple objects in an image from parallax distortion and rapidly matching the image. Brief explanation of the drawing

[0033] FIG. 1 is a block diagram of a multi-object image matching device according to one embodiment of the present invention. FIG. 2 is an example of a similar region according to one embodiment of the present invention. FIG. 3 is a diagram illustrating the creation of a virtual boundary line including all multiple objects according to an embodiment of the present invention. FIG. 4 is a flowchart of a multi-object inclusion image matching method based on the shortest error boundary according to an embodiment of the present invention. Specific details for implementing the invention

[0034] Hereinafter, an image matching device and method including multiple objects according to an embodiment of the present invention will be described in detail with reference to the attached drawings. In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of explanation. Furthermore, the terms described below are defined considering their functions in the present invention, and these may vary depending on the intention or convention of the user or operator. Therefore, the definitions of these terms should be based on the content throughout this specification.

[0035] FIG. 1 is a block diagram of an image matching device including multiple objects according to an embodiment of the present invention, FIG. 2 is an example diagram of a similar region according to an embodiment of the present invention, and FIG. 3 is a diagram illustrating the creation of a virtual boundary line including all multiple objects according to an embodiment of the present invention.

[0036] Referring to FIG. 1, a multi-object image matching device according to one embodiment of the present invention includes an overlapping region calculation module (10), an error matrix extraction unit (20), an object extraction unit (30), a weight generation unit (40), and an image blending unit (50).

[0037] The overlapping region calculation module (10) extracts feature vertices from two images to be stitched and groups these feature vertices into a triangle shape by limiting the minimum distance between the feature points. The overlapping region calculation module (10) calculates similar regions of the images to be stitched by comparing the matched triangles, which are groups of these triangle-shaped feature points, and generates an overlapping region image based on the coordinate values ​​of these similar regions.

[0038] That is, the overlapping area calculation module (10) requires location information for the overlapping area between frames obtained through matching using each frame, the feature points of the frame, and the descriptors.

[0039] Here, a feature point is a point that is distinguishable from the background yet easily identifiable between images for image matching. The descriptor is information such as brightness, color, and gradient direction around the feature point, and is used for image matching.

[0040] The duplicate area calculation module (10) calculates the duplicate area through frame and location information and separates only the corresponding duplicate area.

[0041] The overlapping region calculation module (10) includes a feature point extraction unit (11), a feature point grouping unit (12), a matching pair comparison unit (13), a similar region setting unit (14), and an overlapping region generation unit (15).

[0042] The feature point extraction unit (11) extracts feature points from two images, left and right. The feature point extraction unit (11) can extract points located in positions that are easy to identify in the images as feature points.

[0043] The number of extracted feature points can be set in advance and is not specifically limited. Additionally, the number of extracted feature points can generally be proportional to the size of the target image on which the calculation is performed. For example, the larger the image size, the greater the number of feature points that can be extracted.

[0044] The feature point extraction unit (11) prevents multiple feature points from being extracted in a specific area when extracting feature points. That is, the feature point extraction unit (11) extracts feature points sequentially starting from those with high priority when extracting feature points. If a new feature point is extracted within a set distance from an already extracted feature point, for example, within a set distance from an extracted feature point, the feature point extraction unit (11) may exclude the newly extracted feature point from the calculation without using it.

[0045] The feature point extraction method of the feature point extraction unit (11) can be any method used in image matching techniques to perform feature point extraction, and is not specifically limited.

[0046] The feature point grouping unit (12) can group two other feature points located close to a feature point based on the pixel coordinate value of one feature point for each feature point extracted by the feature point extraction unit (11) into a triangle. The grouped triangle can be called a matched triangle.

[0047] The distance between feature points of a matched triangle can be calculated based on the pixel coordinate values ​​of the feature points. For example, if one of a number of feature points is called a reference feature point and other surrounding feature points are called comparison feature points, a matched triangle consisting of a reference feature point and two other feature points can be generated by detecting the two feature points with the smallest square root value of the 'difference in x-coordinate values ​​+ difference in y-coordinate values' between the reference feature point and the comparison feature points.

[0048] The feature point grouping unit (12) can perform the process of matching each feature point to the aforementioned matched triangle for all previously extracted specific points. Thus, all extracted feature points can be included in at least one matched triangle.

[0049] Meanwhile, if there are three or more detected feature points, two feature points may be finally detected based on user input or pre-set conditions.

[0050] The matching pair comparison unit (13) can compare the matched triangle pairs existing in each target image for determining image similarity regions and match the matched triangle pairs determined to exist in the same location. This process can be performed for all matched triangle pairs.

[0051] The matching pair comparison unit (13) performs the process of comparing the aforementioned matching pair of triangles using the triangle similarity conditions. For example, the matching pair comparison unit (13) can determine whether there is similarity by comparing the ratio of the lengths of each side of the matching pair of triangles to be compared.

[0052] In this case, the threshold value for determining whether similarity exists can be placed in parallel with triangle matching pair comparison algorithms having different threshold values ​​and used for calculating similar regions.

[0053] As shown in FIG. 2, the similarity area setting unit (14) sets a minimum-sized rectangle containing all matched triangle pairs matched by the matched pair comparison unit (13) as the image similarity area (114a).

[0054] The similar region (114a) can be determined using the extreme values ​​of pixel coordinates among the coordinates included in the matched pair of aligned triangles. Here, the extreme values ​​of pixel coordinates may refer to maximum or minimum values. By performing the above process, the location of the similar region (114a) between images can be identified.

[0055] Meanwhile, the similar area between images (114a) may include not only a rectangular shape but also a polygonal shape that can be represented by multiple sides.

[0056] In this embodiment, the similarity area setting unit (14) compares the size of the similarity area (114a) among the result values ​​calculated through each triangle matching pair comparison algorithm arranged in parallel in the matching pair comparison unit (13), and selects the result with the most similar size of the similarity area (114a) as the final similarity area (114a).

[0057] The overlapping area generation unit (15) generates an overlapping area image (115a) by applying the coordinate values ​​of a similar area set by the similar area setting unit (14) to the input image.

[0058] The error matrix extraction unit (20) generates an error matrix from the duplicate area image generated by the duplicate area calculation module (10).

[0059] The object extraction unit (30) detects the movement of multiple objects in the overlapping area image (115a) generated by the overlapping area calculation module (10) and generates a virtual boundary line that includes all the detected multiple objects.

[0060] Generally, when capturing images horizontally, parallax occurs between each capturing device, causing objects that are close to the viewpoint to appear as two. In this case, if a minimum error boundary is generated between objects, the objects in the composite image are damaged.

[0061] Furthermore, conventional object detection techniques in the field of minimum error boundary-based image registration are capable of detecting only a single object, so they have low consistency for multiple objects.

[0062] Accordingly, the object extraction unit (30) detects the movement of an object through the difference of consecutive frames and transmits it as a weight.

[0063] When using an algorithm that considers only the bottom three pixels to generate the existing Minimum Error Boundary (MES), if a boundary value as shown in Fig. 3 is used, the error seam may encroach upon the object and cause an error.

[0064] Accordingly, the object extraction unit (30) generates a virtual boundary line that includes all multiple objects and transmits it as a weight.

[0065] As illustrated in FIG. 3(a), the object extraction unit (30) generates a first line (line α) by connecting the top pixel and the leftmost pixel among the pixels of the object, and generates a second line (line β) by connecting the bottom pixel and the leftmost pixel. The method for generating the first line (line α) is the same as Equation 1 in the form of a linear function.

[0066]

[0067] Here, x1 is the x-coordinate of the top pixel, and y1 is the y-coordinate of the top pixel. x2 is the x-coordinate of the leftmost pixel, and y2 is the y-coordinate of the leftmost pixel.

[0068] Next, as illustrated in FIG. 3(b), the object extraction unit (30) calculates the distance between each edge pixel and the first line (line α) and the second line (line β) for all edge pixels outside the first line (line α) and the second line (line β). At this time, the search area is given by Equation 2, and the distance between each outer boundary pixel and the line is given by Equation 3.

[0069]

[0070]

[0071]

[0072]

[0073] Next, as illustrated in (c) of FIG. 3, the object extraction unit (30) finds the pixel at the farthest distance from the first line (line α) and the second line (line β), respectively, and moves the first line (line α) and the second line (line β) horizontally along the Y-axis to the pixel at the farthest distance.

[0074] Finally, as illustrated in (d) of FIG. 3, the object extraction unit (30) connects the leftmost pixels of the first line (line α) and the second line (line β) that have moved horizontally along the Y-axis, respectively.

[0075] The weight generation unit (40) generates a minimum error boundary-based weight (HVW) in which the value increases as the difference along the x-axis increases, based on the location information of the previous error analysis in the overlapping area image generated by the overlapping area calculation module (10). In this case, the weight generation unit (40) can apply an algorithm such as the minimum error boundary-based weight (HVW) in which the value increases as the difference along the x-axis increases.

[0076] The image blending unit (50) aligns the two left and right images by blending the images using an error matrix generated by the error matrix extraction unit (20), boundary values ​​obtained through object extraction generated by the object extraction unit (30), and minimum error boundary-based weights generated by the weight generation unit (40).

[0077] First, the image blending unit (50) calculates an error value through Equation 4 using an error matrix generated by the error matrix extraction unit (20), a boundary value obtained through object extraction generated by the object extraction unit (30), and a minimum error boundary-based weight generated by the weight generation unit (40).

[0078]

[0079] Here, i and j are pixel positions on the vertical and horizontal axes, respectively, within the overlapping region of the two input images. e(i,j) is the error value, E(i,j) is the error matrix, and edgevalue is the boundary value, and HVW(ij) is the minimum error boundary-based weight.

[0080] Next, the image blending unit (50) generates a boundary line weight matrix using the above-mentioned error value through mathematical formula 5.

[0081]

[0082] Here, S(i,j) is the boundary weight matrix. The boundary weight matrix S(i,j) is generated by summing the lowest value among the top three matrix values ​​in the y-axis direction of the error value e(i,j). Conceptually, this involves summing the minimum errors from the top down to the bottom and selecting the lowest value.

[0083] First, start with the case where i of the error value e(i,j) is 1. At this point, add the smallest value among the values ​​to the left, original position, and right in the j direction from the position where i is 0 to the bottom position where i is 1. Repeat the above algorithm until the bottom position is reached.

[0084] As mentioned above, this process spreads the error from the top downwards. Since the lowest error value is selected, the lowest weighted sum of errors exists at the very bottom. In other words, by following the lowest weighted sum upwards from the bottom, the minimum error boundary MES (Motion Etsimation Set) (j) can be obtained.

[0085] The minimum error boundary MES(j) is a matrix representing the boundary line between two images on the Y-axis, as shown in Equation 6 below.

[0086]

[0087] Next, the image blending unit (50) aligns the left and right images using the above-mentioned minimum error boundary MES(j).

[0088] A method for matching images including multiple objects based on a minimum error boundary according to an embodiment of the present invention will be described in detail below with reference to FIG. 4.

[0089] FIG. 4 is a flowchart of a minimum error boundary-based multi-object image matching method according to an embodiment of the present invention.

[0090] First, the feature point extraction unit (11) extracts feature points from two images, left and right (S10). In this case, the feature point extraction unit (11) extracts feature points sequentially starting from those with a high priority when extracting feature points. If a new feature point is extracted within a set distance from an already extracted feature point, the newly extracted feature point is not used and is excluded from the calculation.

[0091] The feature point grouping unit (12) creates a matched triangle for each feature point extracted by the feature point extraction unit (11) by grouping two other feature points located close to the feature point based on the pixel coordinate value of one feature point. This process of creating a matched triangle is performed for all extracted specific points. Therefore, all extracted feature points can be included in at least one matched triangle.

[0092] Next, the matching pair comparison unit (13) compares the matching triangle pairs existing in each image to perform image similarity region determination and matches the matching triangle pairs that are determined to exist in the same location (S30).

[0093] At this time, the matching pair comparison unit (13) performs the process of comparing the matching pair triangles mentioned above using the triangle similarity condition.

[0094] Next, the similarity area setting unit (14) sets a rectangle of the minimum size that includes all matched triangle pairs matched by the matched pair comparison unit (13) as the image similarity area (114a) (S40).

[0095] Next, the overlapping area generation unit (15) generates an overlapping area image (115a) by applying the coordinate values ​​of the similar area set by the similar area setting unit (14) to the input image (S50).

[0096] The error matrix extraction unit (20) generates an error matrix from the duplicate region image generated by the duplicate region calculation module (10) (S60).

[0097] Additionally, the object extraction unit (30) detects the movement of multiple objects in the overlapping area image generated by the overlapping area calculation module (10) and generates a virtual boundary line that includes all the detected multiple objects (S70). In this case, the object extraction unit (30) generates a virtual boundary line that includes all the multiple objects and transmits it as a weight. In this case, the object extraction unit (30) generates a first line (line α) by connecting the top pixel and the leftmost pixel among the pixels of the objects, and generates a second line (line β) by connecting the bottom pixel and the leftmost pixel. For all edge pixels outside the first line (line α) and the second line (line β), the object extraction unit (30) calculates the distance between each edge pixel and the first line (line α) and the second line (line β), respectively. The object extraction unit (30) finds the pixel that is at the farthest distance from the first line (line α) and the second line (line β), respectively, moves the first line (line α) and the second line (line β) horizontally along the Y-axis, and then connects the leftmost pixel of the first line (line α) and the second line (line β) that has been moved horizontally along the Y-axis.

[0098] Additionally, the weight generation unit (40) generates a minimum error boundary-based weight (HVW) in which the value increases as the difference along the x-axis increases in the overlapping region image generated by the overlapping region calculation module (10) (S80). In this case, the weight generation unit (40) can apply an algorithm such as the minimum error boundary-based weight (HVW) in which the value increases as the difference along the x-axis increases.

[0099] As described above, as the error matrix generated by the error matrix extraction unit (20), the boundary line value through object extraction generated by the object extraction unit (30), and the minimum error boundary-based weight generated by the weight generation unit (40) are generated, the image blending unit (50) blends the images using the error matrix, the boundary line value, and the minimum error boundary-based weight to align the two left and right images.

[0100] First, the image blending unit (50) calculates an error value using an error matrix generated by the error matrix extraction unit (20), a boundary value obtained through object extraction generated by the object extraction unit (30), and a minimum error boundary-based weight generated by the weight generation unit (40), and generates a boundary weight matrix using this error value through Equation 5. Based on this boundary weight matrix, a minimum error boundary MES is generated.

[0101] Next, the image blending unit (50) aligns the two left and right images using the above-mentioned minimum error boundary MES(j) and performs blending processing (S90).

[0102] As such, the image matching device and method based on a minimum error boundary of a multi-object image according to one embodiment of the present invention prevents damage to the time invariance of moving multi-objects when matching images captured in a horizontal direction, enables shortening the image matching speed, and can provide a stitched image that is more suitable for surveillance environments where various objects appear frequently in the image and rapid image delivery to the image observer is required.

[0103] In addition, the image matching device and method based on a minimum error boundary of the present invention can determine similar regions with high accuracy by limiting the extraction of adjacent feature points when extracting feature points, and can eliminate repeatability in determining similar regions by processing threshold values ​​for determining similar regions in parallel.

[0104] Furthermore, the image matching device and method based on a minimum error boundary of an image containing multiple objects according to one embodiment of the present invention can increase consistency by acquiring a motion image through the difference between consecutive frames of an image and transmitting it in the form of a weight to accurately detect the boundary lines of multiple objects.

[0105] The implementations described herein may be implemented, for example, as methods or processes, devices, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., devices or programs). Devices may be implemented in appropriate hardware, software, and firmware, etc. Methods may be implemented in devices such as processors, which generally refer to processing devices including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices such as computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end-users.

[0106] Although the present invention has been described with reference to the embodiments illustrated in the drawings, this is merely illustrative and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the claims below. Explanation of the symbols

[0107] 10: Overlapping Region Calculation Module 11: Feature Point Extraction Unit 12: Feature point grouping section 13: Matching pair comparison section 14: Similar Area Setting Section 15: Overlapping Area Creation Section 20: Error Matrix Extraction Unit 30: Object Extraction Unit 40: Weight generation unit 50: Image blending unit

Claims

Claim 1 An overlapping area calculation module that extracts feature points from two images, groups the feature points into a triangular shape to generate a matched triangle, compares the matched triangle to calculate a similar area between the two images based on the comparison result, and then generates an overlapping area image based on the coordinate values ​​of the similar area; an error matrix extraction unit that generates an error matrix from the overlapping area image; an object extraction unit that detects the movement of multiple objects in the overlapping area image and generates a virtual boundary line that includes all detected multiple objects; and a weight generation unit that generates a minimum error boundary-based weight in which the value increases as the difference along the x-axis from the boundary line value of the previous image detected in the overlapping area image increases. A multi-object inclusion image matching device comprising: an image blending unit that calculates a Minimum Error Seam (MES) using the error matrix, the boundary line, and the minimum error seam-based weights, and then aligns and blends two images based on the minimum error seam MES; wherein the object extraction unit connects a first line connecting the top pixel and the leftmost pixel among the edge pixels in the overlapping area image and a second line connecting the bottom pixel and the leftmost pixel, finds the pixel at the relative distance from each of the first line and the second line, moves the first line and the second line horizontally along the Y-axis, and then connects the leftmost pixels of the first line and the second line respectively to generate the boundary line that includes all of the multiple objects. Claim 2 In claim 1, the overlapping region calculation module comprises: a feature point extraction unit for extracting the feature points; a feature point grouping unit for grouping each of the feature points into triangles to generate the matched triangles; a matched pair comparison unit for comparing the matched triangle pairs with each other and matching the matched triangle pairs determined to exist in the same location; a similar region setting unit for setting a minimum-sized rectangle containing all the matched triangle pairs as the similar region; and an overlapping region generation unit for generating the overlapping region image using the coordinate values ​​of the similar region, characterized in that the multi-object including image matching device. Claim 3 A multi-object image matching device according to claim 2, characterized in that the number of feature points is proportional to the size of the image. Claim 4 A multi-object image matching device according to claim 2, wherein the feature point extraction unit extracts the feature points according to priority, and excludes any one of the feature points depending on whether the distance between the feature points is within a preset distance. Claim 5 A multi-object image matching device according to claim 2, wherein the feature point grouping unit groups the three feature points located at the relatively closest positions among the feature points to generate the matched triangle. Claim 6 A multi-object image matching device according to claim 2, wherein the matching pair comparison unit performs the process of comparing matching pair triangle pairs using triangle similarity conditions. Claim 7 A multi-object image matching device according to claim 6, wherein the matching pair comparison unit determines whether the matched triangles are similar by comparing the length ratios of each side of the matched triangle pairs. Claim 8 A multi-object image matching device according to claim 2, wherein the similar area setting unit sets a minimum-sized rectangle containing all of the matched triangle pairs as the similar area. Claim 9 A multi-object image matching device according to claim 2, wherein the similarity region setting unit sets the similarity region using the maximum or minimum value among the coordinates included in the matched triangle pair. Claim 10 delete Claim 11 A multi-object image matching device according to claim 1, wherein the image blending unit generates an error value generated through the error matrix, the boundary line, and the minimum error boundary-based weight, and generates the minimum error boundary by summing the lowest value among the top three matrix values ​​in the y-axis direction of the error value. Claim 12 A step in which an overlapping region calculation module extracts feature points from two images, groups the feature points into a triangular shape to generate a matched triangle, compares the matched triangle to calculate a similar region between the two images based on the comparison result, and then generates an overlapping region image based on the coordinate values ​​of the similar region; a step in which an error matrix extraction unit generates an error matrix from the overlapping region image; a step in which an object extraction unit detects the movement of multiple objects in the overlapping region image and generates a virtual boundary line that includes all detected multiple objects; a step in which a weight generation unit generates a minimum error boundary-based weight in which the value increases as the difference along the x-axis from the boundary line value of the previous image detected in the overlapping region image increases. A method for aligning images containing multiple objects, wherein the image blending unit calculates a Minimum Error Seam (MES) using the error matrix, the boundary line, and the minimum error seam-based weights, and then aligns and blends two images based on the minimum error seam MES, and the step of generating the virtual boundary line is characterized by connecting a first line connecting the topmost pixel and the leftmost pixel among the edge pixels in the overlapping area image and a second line connecting the bottommost pixel and the leftmost pixel, finding the pixel at the relative distance from each of the first line and the second line, moving the first line and the second line horizontally along the Y-axis, and then connecting the leftmost pixels of the first line and the second line respectively to generate the boundary line that includes all of the multiple objects. Claim 13 In claim 12, the step of generating the overlapping region image comprises: a step of extracting the feature points; a step of grouping each of the feature points into triangles to generate the matched triangles; a step of comparing the matched triangle pairs with each other to match the matched triangle pairs determined to exist in the same location; a step of setting a minimum-sized rectangle containing all the matched triangle pairs as the similar region; and a step of generating the overlapping region image using the coordinate values ​​of the similar region. Claim 14 A multi-object image matching method according to claim 13, characterized in that the number of feature points is proportional to the size of the image. Claim 15 A method for matching images including multiple objects according to claim 13, wherein the step of extracting the feature points is characterized by extracting the feature points according to priority, and excluding any one of the feature points depending on whether the distance between the feature points is within a preset distance. Claim 16 In claim 13, the step of grouping each of the feature points into a triangle to generate the matched triangle is characterized by grouping the three feature points located at the relatively closest positions among the feature points to generate the matched triangle, in a multi-object image matching method. Claim 17 In claim 13, the step of matching pairs of matched triangles determined to exist in the same location by comparing the matched pairs of triangles with each other is characterized by performing the process of comparing the matched pairs of triangles using triangle similarity conditions. Claim 18 In claim 17, the step of matching pairs of matched triangles determined to exist in the same location by comparing the matched pairs of triangles with each other is characterized by determining whether the matched triangles are similar by comparing the ratio of the lengths of each side of the matched pairs of triangles. Claim 19 In claim 13, the step of setting a minimum-sized rectangle containing all of the matched triangle pairs as the similar region is characterized by setting a minimum-sized rectangle containing all of the matched triangle pairs as the similar region. Claim 20 A method for image matching including multiple objects according to claim 13, wherein the step of setting a minimum-sized rectangle containing all of the matched triangle pairs as the similar region is characterized by setting the similar region using the maximum or minimum value among the coordinates included in the matched triangle pairs. Claim 21 delete Claim 22 A method for matching images containing multiple objects according to claim 12, wherein the step of matching and blending two images based on the minimum error boundary MES comprises generating an error value through the error matrix, the boundary line, and the minimum error boundary-based weights, and generating the minimum error boundary by summing the lowest value among the top three matrix values ​​in the y-axis direction of the error value.

Images