System and method for determining object displacement on basis of image analysis of object, and marker module

Abstract

Provided is a method for determining object displacement on the basis of image analysis of an object. The method comprises the steps of: acquiring captured images of a first marker unit over a plurality of frames; acquiring a one-dimensional pixel value distribution along a first reference line from a reference frame and a target frame among the plurality of frames; and determining a first direction displacement of a target object in the target frame with respect to the reference frame on the basis of a difference between the locations of a first direction representative value of the reference frame and a first direction representative value of the target frame.

Description

System and method for determining object displacement based on image analysis of the object and marker module

[0001] The present invention relates to determining object displacement, and more particularly, to a system and method for determining object displacement based on image analysis of an object and detecting and notifying the occurrence of abnormal displacement, and a marker module that can be used therefor.

[0002] Methods for measuring the displacement of a target object are required for various purposes, such as predicting or assessing the structural safety of an object. Conventionally, methods utilizing various physical sensors directly attached to the object have been primarily used to measure the displacement of a target object. However, these types of sensors require excessive installation work, have limited displacement measurement ranges, and incur high costs for sensor installation.

[0003] Therefore, instead of attaching a sensor to an object, a method has been proposed to acquire an image of the object at a desired point in time and determine the displacement of the object by comparing it with the image of the object at a reference point in time. However, even in this image analysis-based object displacement measurement, in order to secure higher precision, an imaging device with higher resolution must be equipped, which is not only disadvantageous in terms of budget, but also has the limitation that even images obtained by an imaging device with high resolution cannot detect object movement below the pixel value.

[0004] Multiple image processing algorithms have been proposed to improve the accuracy of object displacement based on image analysis, or methods that utilize more surrounding information. However, image processing techniques such as the Digital Image Correlation (DIC) technique have problems with the computational load of the processor, which excessively increases the performance requirements of the analysis equipment and requires a certain amount of processing time for displacement measurement, making it difficult to implement real-time analysis.

[0005] An object of the present invention to solve the above-described problem is to provide a method for determining object displacement based on image analysis of an object and detecting and notifying when an abnormal displacement occurs, wherein a photographed image of a marker having a central strip and a background portion arranged on both sides of the central strip is obtained from an image capturing unit located on a rigid object separated from the object, and a data processing unit applies numerical analysis to the pixel value distribution to derive a representative value, thereby enabling high precision in measuring the displacement of an object through fast calculation without utilizing a high-resolution image.

[0006] Another object of the present invention to solve the above-described problem is to provide a marker module capable of measuring the displacement of an object with high precision through fast calculation without utilizing a high-resolution image by obtaining a photographed image of a marker having a central strip and a background portion arranged on both sides of the central strip from an image capturing unit located on a rigid object separated from the object, and applying numerical analysis to the pixel value distribution in a data processing unit to derive a representative value.

[0007] However, the problem to be solved by the present invention is not limited to this, and may be expanded in various ways without departing from the spirit and scope of the present invention.

[0008] According to one embodiment of the present invention for achieving the above-described object, a method for determining object displacement based on image analysis of an object comprises the steps of: obtaining a photographed image of a first marker portion over a plurality of frames from an image capturing unit positioned on a rigid object spaced apart from the object, the first marker portion having a first central strip arranged in a first direction and a first background portion arranged on both sides of the first central strip, and being attached to a target object; obtaining, by a data processing unit, a one-dimensional pixel value distribution along a first reference line arranged in a second direction orthogonal to the first direction on the first marker portion in a reference frame among the plurality of frames, and obtaining, in a target frame among the plurality of frames, a one-dimensional pixel value distribution along the first reference line; And a step of determining a first direction displacement of a target object in a target frame with respect to the reference frame based on a positional difference between a first direction representative value according to a one-dimensional pixel value distribution according to a first reference line of the reference frame and a first direction representative value according to a one-dimensional pixel value distribution according to a first reference line of the target frame; a step of detecting and notifying when an abnormal displacement occurs;

[0009] According to one aspect, the step of acquiring the captured image from the video capturing unit located on a rigid object spaced apart from the object further acquires a captured image for a second marker unit over a plurality of frames, the second marker unit having a second central strip disposed in the second direction and a second background unit disposed on both sides of the second central strip, and is attached to a target object, and the step of acquiring the pixel value distribution further acquires, in a reference frame among the plurality of frames, a one-dimensional pixel value distribution along a second reference line disposed in a first direction on the second marker unit, and in the target frame, a one-dimensional pixel value distribution along the second reference line, and the step of determining the displacement further determines a second direction displacement of the target object in the target frame with respect to the reference frame based on a positional difference between a second direction representative value according to the one-dimensional pixel value distribution along the second reference line of the reference frame and a second direction representative value according to the one-dimensional pixel value distribution along the second reference line of the target frame. Can be configured.

[0010] According to one aspect, the first marker unit and the second marker unit can be attached to the measurement point of the target object while being spaced apart from each other by a predetermined distance or more.

[0011] According to one aspect, the first direction representative value according to the one-dimensional pixel value distribution according to the first reference line may be a point corresponding to the maximum point of the one-dimensional pixel value distribution.

[0012] According to one aspect, the step of determining the displacement may include the step of performing a Fast Fourier Transform (FFT) on the one-dimensional pixel value distribution, extracting a plurality of valid components from the result of performing the FFT, and formulating the one-dimensional pixel value distribution based on at least one of a magnitude, a wavenumber, or a phase of the extracted valid components.

[0013] According to one aspect, the step of determining the displacement may determine a maximum point for the one-dimensional pixel value distribution using numerical analysis based on a formula for the one-dimensional pixel value distribution.

[0014] According to one aspect, the step of determining the displacement may include performing an area comparison between two sub-regions divided based on the midpoints of two arbitrary points using a formula for the one-dimensional pixel value distribution, and determining the maximum point for the one-dimensional pixel value distribution by updating the maximum point to the midpoints of both ends of the sub-region having the larger area.

[0015] According to one aspect, the step of determining the displacement may determine a maximum point for the one-dimensional pixel value distribution based on a bisection method.

[0016] According to one aspect, at least a portion of the first central strip may be formed of a reflector that reflects light, and at least one of a point reflector or a point light source may be arranged at at least one of the corners of the first marker portion.

[0017] According to one aspect, the method may further include a step of calculating an evaluation value for structural safety of the target object in the target frame by using a difference between a first direction displacement of the target object determined based on the first marker portion in the target frame and a second direction displacement of the target object determined based on the second marker portion.

[0018] According to one aspect, the method may further include a step of calculating at least one of a distance direction displacement of the target object in the target frame with respect to the reference frame, a tilt direction of the target object, or a tilt degree of the target object, based on at least one of a second direction length of the first central strip in the target frame or a first direction length of the second central strip;

[0019] According to one aspect, the photographed image for the first marker unit can be acquired based on a camera mounted on an unmanned aerial vehicle.

[0020] In order to solve the above-described problem, according to another embodiment of the present invention, a marker module attached to a target object for measuring a displacement of the target object, the marker module includes a first marker portion, the first marker portion having a first central strip arranged in a first direction; and a first background portion arranged on both sides of the first central strip; wherein the marker module acquires a photographed image of the first marker portion over a plurality of frames; acquires a one-dimensional pixel value distribution along a first reference line arranged in a second direction orthogonal to the first direction on the first marker portion in a reference frame among the plurality of frames; and acquires a one-dimensional pixel value distribution along the first reference line in a target frame among the plurality of frames. And, based on the positional difference between the first direction representative value according to the one-dimensional pixel value distribution along the first reference line of the reference frame and the first direction representative value according to the one-dimensional pixel value distribution along the first reference line of the target frame, it can be used to determine the first direction displacement of the target object in the target frame with respect to the reference frame.

[0021] A system for determining displacement of an object based on image analysis of an object according to another embodiment of the present invention comprises: an image capturing unit for acquiring a captured image of a first marker portion over a plurality of frames, the first marker portion having a first central strip disposed in a first direction and a first background portion disposed on both sides of the first central strip, and attached to a target object; and a data processing unit configured to determine displacement of the object based on the captured image, wherein the data processing unit acquires, in a reference frame among the plurality of frames, a one-dimensional pixel value distribution along a first reference line disposed in a second direction orthogonal to the first direction on the first marker portion, and acquires, in a target frame among the plurality of frames, a one-dimensional pixel value distribution along the first reference line; And it can be configured to determine a first direction displacement of a target object in the target frame with respect to the reference frame based on a positional difference between a first direction representative value according to a one-dimensional pixel value distribution according to a first reference line of the reference frame and a first direction representative value according to a one-dimensional pixel value distribution according to a first reference line of the target frame.

[0022] The disclosed technology may have the following effects. However, this does not mean that a particular embodiment must include all or only the following effects, and thus the scope of the disclosed technology should not be construed as being limited thereby.

[0023] According to a method or marker module for determining object displacement based on image analysis of an object according to an embodiment of the present invention described above and detecting and notifying when an abnormal displacement occurs, a photographed image of a marker having a central strip and a background portion arranged on both sides of the central strip is secured, and a representative value is derived by applying numerical analysis to the pixel value distribution, thereby enabling the displacement of an object to be measured with high precision through fast calculation without utilizing a high-resolution image.

[0024] That is, since it does not handle high-resolution images by utilizing simple formulas, fast calculations are possible. Therefore, when measuring motion in an image, it is possible to measure the displacement of an object with high accuracy while using fewer processing resources than when measuring the displacement of an object using conventional image processing methods by utilizing mathematical formulas and the bisection method for the data format without using conventional image processing methods such as DIC (Digital Image Correlation), Optical Flow, and Phase Flow.

[0025] FIG. 1 is a drawing showing a system for measuring displacement of a bridge, which is an example of an object according to one embodiment of the present invention.

[0026] Figure 2 is an exemplary diagram of a marker module according to one embodiment of the present invention.

[0027] Figure 3 illustrates the determination of the reference position in the x-axis direction according to the marker module of Figure 2.

[0028] Figure 4 illustrates the determination of the reference position in the y-axis direction according to the marker module of Figure 1.

[0029] Figure 5 illustrates a formulating procedure for a one-dimensional pixel value distribution obtained from a captured image.

[0030] Figure 6 illustrates the procedure for determining the maximum point location based on the bisection method using a formula.

[0031] FIG. 7 is an exemplary diagram of a marker module having a reflector or light source according to one embodiment of the present invention.

[0032] FIG. 8 is a conceptual diagram for detecting distance direction displacement or tilting according to one embodiment of the present invention.

[0033] FIG. 9 is a flowchart of a method for determining object displacement based on image analysis of an object according to one embodiment of the present invention.

[0034] FIG. 10 is an exemplary diagram of a structural safety evaluation procedure of a target object according to one aspect of the present invention.

[0035] FIG. 11 is an exemplary diagram of a procedure for detecting distance direction displacement and tilting according to one aspect of the present invention.

[0036] FIG. 12 is a block diagram showing an exemplary configuration of a computing system in which methods according to one embodiment of the present invention can be performed.

[0037] The present invention can be modified in various ways and has various embodiments, and specific embodiments are illustrated in the drawings and described in detail.

[0038] However, this is not intended to limit the present invention to a specific embodiment, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

[0039] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes any combination of multiple related items described herein or any item among multiple related items described herein.

[0040] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0041] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0042] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0043] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate an overall understanding in describing the present invention, identical reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0044]

[0045] As previously discussed, methods for measuring the displacement of a target object are required for various reasons. For example, to predict or assess the structural safety of an object, detecting the direction and extent of displacement from a reference position can be a crucial indicator.

[0046] It should be noted that the method for determining object displacement based on image analysis of an object according to an embodiment of the present invention described in this disclosure may be used, for example, to perform an evaluation of the structural safety of an object, but is not limited thereto. That is, a person having ordinary skill in the art related to measuring object displacement will understand that the method for determining object displacement based on image analysis of an object according to an embodiment of the present disclosure may be adopted as a utilization for any purpose for measuring the displacement of an object.

[0047]

[0048] In the past, a method was mainly used to measure the displacement of a target object by using various physical sensors directly attached to the object. However, this method had the disadvantage of requiring excessive man-hours for installing sensors using this type of attachment, and also incurring high costs for constructing sensor facilities.

[0049] Therefore, instead of attaching a sensor to an object, a method has been proposed to acquire an image of the object at a desired point in time and determine the displacement of the object by comparing it with the image of the object at a reference point in time. However, even in this image analysis-based object displacement measurement, in order to secure higher precision, an imaging device with higher resolution must be equipped, which is not only disadvantageous in terms of budget, but also has the limitation that even images obtained by an imaging device with high resolution cannot detect object movement below the pixel value.

[0050] Multiple image processing algorithms have been proposed to improve the accuracy of object displacement based on image analysis, or methods that utilize more surrounding information. However, image processing techniques such as the Digital Image Correlation (DIC) technique have problems with the computational load of the processor, which excessively increases the performance requirements of the analysis equipment and requires a certain amount of processing time for displacement measurement, making it difficult to implement real-time analysis.

[0051] A method for determining object displacement based on image analysis of an object according to one embodiment of the present invention is intended to solve such a problem, and comprises obtaining a photographed image of a marker having a central strip and a background portion arranged on both sides of the central strip from an image capturing unit located on a rigid object apart from the object, applying numerical analysis to a pixel value distribution to derive a representative value, thereby enabling measurement of object displacement with high precision through fast calculation without utilizing a high-resolution image.

[0052] That is, since it does not handle high-resolution images by utilizing simple formulas, fast calculations are possible. Therefore, when measuring motion in an image, it is possible to measure the displacement of an object with high accuracy while using fewer processing resources than when measuring the displacement of an object using conventional image processing methods by utilizing mathematical formulas and the bisection method for the data format without using conventional image processing methods such as DIC (Digital Image Correlation), Optical Flow, and Phase Flow.

[0053]

[0054] In relation to this, FIG. 1 is a drawing showing a system for determining displacement of a bridge, which is an example of an object according to one embodiment of the present invention, and detecting and notifying the occurrence of abnormal displacement.

[0055] As illustrated in FIG. 1, a system (10) for determining object displacement according to one aspect of the present invention includes an image capturing unit (20), markers (25-1, 25-3, 25-5), a data processing unit (30), and a communication unit (40). In addition, an administrator terminal (50) that can be used by an administrator may be included. In FIG. 1, an embodiment in which markers are installed at three locations, such as a first marker (25-1), a second marker (25-3), and a third marker (25-5), is illustrated as an example, but the technical idea of ​​the present invention is not limited thereto, and the markers may be installed at one location or two or more locations. According to one aspect, the displacement magnitude can generally be measured at a location where the maximum displacement is expected. Therefore, according to one aspect, the markers can generally be installed at a location where the maximum displacement is expected.

[0056] The video recording unit (20) is configured to acquire images of the markers (25-1, 25-3, 25-5). According to one aspect, the video recording unit (20) may be a camera, but is not limited thereto.

[0057] The data processing unit (30) can determine the displacement of the object and / or marker (25-1, 25-3, 25-5) based on the images of the object and / or marker (25-1, 25-3, 25-5) acquired from the video capturing unit (20). That is, the data processing unit (30) can obtain, store, and digitize displacement measurement values ​​based on the images.

[0058] The communication unit (40) can transmit numerical information on such displacement measurement values ​​to the manager terminal (50) using wired or wireless communication technology.

[0059] Accordingly, the administrator can check information based on the administrator terminal (50). According to one aspect, when an abnormal displacement occurs in an object, the administrator can more easily monitor the object by detecting this and transmitting an abnormal condition occurrence message to the administrator terminal (50).

[0060] In relation to this, FIG. 2 is an exemplary diagram of a marker module according to one embodiment of the present invention, and FIG. 9 is a flowchart of a method for determining object displacement based on image analysis of an object according to one embodiment of the present invention. Hereinafter, a method for determining object displacement based on image analysis of an object according to one embodiment of the present invention will be described in more detail with reference to FIG. 2 and FIG. 9.

[0061] As illustrated in FIG. 9, according to a method for determining object displacement based on image analysis of an object according to one embodiment of the present invention, first, a photographed image of a first marker portion (110) may be acquired over a plurality of frames (step 810). Here, the first marker portion (110) may include a first central strip (111) arranged in a first direction (e.g., y-axis direction) and first background portions (113-1, 113-2) arranged on both sides of the first central strip (111), and may be attached to a target object.

[0062] According to one aspect, the step of acquiring the photographed image (step 810) may further acquire a photographed image for the second marker portion (120) over a plurality of frames. Here, the second marker portion (120) may include a second central strip (121) arranged in a second direction (e.g., in the x-axis direction) and second background portions (123-1, 123-2) arranged on both sides of the second central strip (121), and may be attached to a target object.

[0063] According to one aspect, the captured image for the first marker unit (110) and / or the second marker unit (120) may be obtained based on a camera mounted on an unmanned aerial vehicle, but is not limited thereto.

[0064]

[0065] In relation to this, FIG. 1 illustrates a system for determining displacement of a bridge, which is an example of an object according to an embodiment of the present invention, and detecting and notifying the occurrence of abnormal displacement. Although FIG. 1 is illustrated in the form of a bridge as an example, the technical idea of ​​the present invention is not limited thereto, and the object to be determined for displacement may be any one of arbitrary objects. A non-limiting but more specifically, object displacement measurement system (10) according to one aspect will be described. An image capturing unit (20) may include an image capturing camera and a camera support. For example, the camera support may be configured to measure object displacement by firmly mounting a tripod and the tripod to a fixed object. According to one aspect, at least one of a vibration damping device or a weight may be included to prevent measurement errors and / or errors due to interference from wind or traffic. The camera support is not limited to a tripod. According to one aspect, the system (10) includes an object displacement measurement unit (25-1, 25-3, 25-5). For example, the object displacement measurement unit (25-1, 25-3, 25-5) may include the markers illustrated in FIG. 2. The marker shape is not limited to a square and may include geometric shapes such as a triangle, a circle, and an oval. According to one aspect, the data processing unit (30) is connected to the image capturing unit (20) using a wired or wireless communication network, receives object measurement information and motion information, and stores them as big data. In addition, the big data may mean data in which identification information of objects, identification information of markers, displacement information measured over time, etc. are matched and stored. In addition, the data processing unit (30) calculates displacement of the object from changes in object measurement information over time.

[0066] The communication unit (40) can transmit digitized information on such displacement measurement values ​​to the manager terminal (50) using wired or wireless communication technology. Here, it will be apparent to a person having ordinary skill in the art that the wired or wireless communication technology is not limited to a communication architecture having a specific protocol, and any communication method among any information transmission means that enables transmission or reception of data can be adopted.

[0067] By transmitting such measurement information and / or displacement information, the administrator can check the information based on the administrator terminal (50). According to one aspect, when an abnormal displacement occurs in an object, the administrator can more easily monitor the object by detecting this and transmitting an abnormal condition occurrence message to the administrator terminal (50). The administrator device may include any type of computing device among fixed or portable computing devices. That is, for example, the administrator device (50) may be a smart device such as a smartphone or a tablet PC, or may be a traditional computing device such as a server or a personal computer.

[0068] Meanwhile, according to one aspect, the system (10) may further include a power supply (60). For example, the power supply may be used to supply electricity to a light source for tracking the marker (30) even at night according to one aspect of the present invention, but is not limited thereto. The power supply (60) may be, for example, fixed to an object. According to another aspect, instead of having a separate power supply (60), a power supply path may be provided for supplying power from the object to be measured.

[0069] In relation to this, an exemplary marker module (100) is illustrated in FIG. 2. According to one aspect, the marker module (100) may include a first marker portion (110) and / or a second marker portion (120). For example, the first marker portion (110) may be used to detect a second direction movement of a target object to which the marker module (100) is attached, and the second marker portion (120) may be used to detect a first direction movement of a target object to which the marker module (100) is attached, but is not limited thereto. Hereinafter, the first direction may represent, for example, the y-axis direction in the drawing, and the second direction may represent, for example, the x-axis direction in the drawing, but is not limited thereto, and it is to be noted that the first or second direction may represent any different directions.

[0070] As illustrated in FIG. 2, the first marker unit (110) has a first central strip (111) arranged in a first direction (e.g., y-axis direction) and first background units (113-1, 113-2) arranged on both sides of the first central strip (111), and the second marker unit (120) has a second central strip (121) arranged in a second direction (e.g., x-axis direction) different from the first direction and second background units (123-1, 123-2) arranged on both sides of the second central strip (121), so that the first marker unit (110) and the second marker unit (120) can be utilized to detect movement in different directions of a target object to which the marker module (100) is attached. According to one aspect, the first marker unit (110) and the second marker unit (120) can be attached to the measurement point of the target object at a distance from each other greater than a predetermined distance, but are not limited thereto.

[0071]

[0072] As exemplarily illustrated in FIG. 2, the first central strip (111) and / or the second central strip (121) may be white, and the first background portion (113-1, 113-2) and / or the second background portion (123-1, 123-2) may be black, but is not limited thereto. The central strip and the background portion may be configured to have visual characteristics that are distinguishable from each other, and may be distinguished in at least one aspect of saturation or brightness, for example. In determining the reference position for the marker portion described below, according to one aspect, in consideration of contrasting properties of the central strip and the background portion, a point having properties closest to the properties of the central strip may be determined as the reference position.

[0073] FIG. 3 illustrates determination of a reference position in the x-axis direction according to the marker module of FIG. 2. In the exemplary marker module (100) of FIG. 2, the reference position in the x-axis direction can be determined with respect to the first marker unit (110). The exemplary first marker unit (110) of FIG. 2 has a first central strip (111) arranged in the y-axis direction and has background units (113-1, 113-2) on both sides. Here, a one-dimensional pixel value distribution can be obtained along a first reference line (115) arranged along the x-axis direction. That is, data of portions corresponding to the first reference line (115) in a photographed image for the first marker unit (110) can be obtained, and the obtained data can be expressed in a one-dimensional form. As illustrated in FIG. 3, the one-dimensional pixel value distribution along the first reference line (115) may have, for example, a graph in the form of a quadratic function, but is not limited thereto. Following the first reference line (115), it starts from the black background portion (113-1) in the exemplary first marker portion of FIG. 2, passes through the white central strip (111), and then passes through the black background portion (113-2), so that the pixel value (which may be referred to as intensity, for example) may have a one-dimensional distribution that gradually rises and then falls again, as illustrated in FIG. 3. Here, by determining the maximum point (117) having the highest pixel value, the maximum point (117) may be determined as the x-axis direction reference position of the first marker portion (110).

[0074] FIG. 4 illustrates determination of a reference position in the y-axis direction according to the marker module of FIG. 2. In the exemplary marker module (100) of FIG. 2, the reference position in the y-axis direction can be determined with respect to the second marker unit (120). The exemplary second marker unit (120) of FIG. 2 has a second central strip (121) arranged in the x-axis direction and has background units (123-1, 123-2) on both sides. Here, a one-dimensional pixel value distribution can be obtained along a second reference line (125) arranged along the y-axis direction. That is, data of portions corresponding to the second reference line (125) in a photographed image for the second marker unit (120) can be obtained, and the obtained data can be expressed in a one-dimensional form. As illustrated in FIG. 4, the one-dimensional pixel value distribution along the second reference line (125) may have, for example, a graph in the form of a quadratic function, but is not limited thereto. Following the second reference line (125), it starts from the black background portion (123-1) in the exemplary second marker portion of FIG. 2, passes through the white central strip (121), and then passes through the black background portion (123-2), so that the pixel value (which may be referred to as intensity, for example) may have a one-dimensional distribution that gradually rises and then falls again, as illustrated in FIG. 4. Here, by determining the maximum point (127) having the highest pixel value, the maximum point (127) may be determined as the y-axis direction reference position of the second marker portion (120).

[0075]

[0076] According to one aspect, a plurality of frames in which a photographed image for the marker module (100) is acquired may include a reference frame and a target frame. The reference frame may be, for example, the first frame of the image, but is not limited thereto. For example, the reference frame may refer to a frame in which a target object to which the marker module (100) is attached is in a normal state. In such a reference frame, a reference position according to a reference line of the marker module (100) is determined, and a motion signal may be acquired by recording the relative position of the maximum point acquired in each subsequent frame with respect to such a reference position. That is, a displacement between the reference position in the reference frame and the reference position in the target frame may be determined as a displacement of the object in the target frame with respect to the reference frame.

[0077]

[0078] The one-dimensional pixel value distribution and displacement determination according to this reference line are described again with reference to Fig. 9.

[0079] Referring to FIG. 9, in a reference frame among the plurality of frames, a one-dimensional pixel value distribution can be obtained along a first reference line (115) arranged in a second direction orthogonal to the first direction on a first marker portion (110), and in a target frame among the plurality of frames, a one-dimensional pixel value distribution can be obtained along the first reference line (115) (step 820).

[0080] According to one aspect, the step of obtaining the pixel value distribution (step 820) may obtain a one-dimensional pixel value distribution along a second reference line (125) arranged in a first direction on the second marker unit (120) in a reference frame among the plurality of frames, and further obtain a one-dimensional pixel value distribution along the second reference line (125) in the target frame.

[0081]

[0082] Referring again to FIG. 9, a first direction displacement of a target object in a target frame with respect to the reference frame can be determined (step 830) based on a positional difference between a first direction representative value (117) according to a one-dimensional pixel value distribution along a first reference line (115) of the reference frame and a first direction representative value (117) according to a one-dimensional pixel value distribution along a first reference line (115) of the target frame.

[0083] According to one aspect, the step of determining the displacement (step 830) may further determine the second direction displacement of the target object in the target frame with respect to the reference frame based on a positional difference between a second direction representative value (127) according to a one-dimensional pixel value distribution along a second reference line (125) of the reference frame and a second direction representative value (127) according to a one-dimensional pixel value distribution along a second reference line (125) of the target frame.

[0084] That is, the displacement of the target object in the target frame with respect to the reference frame can be determined by comparing the reference position with respect to the reference frame and the reference position with respect to the target frame. As discussed above, according to one aspect, the first direction representative value according to the one-dimensional pixel value distribution along the first reference line may be a point corresponding to the maximum point of the one-dimensional pixel value distribution, but it should be noted that the present invention is not limited thereto.

[0085]

[0086] Fig. 5 describes a formulating procedure for a one-dimensional pixel value distribution obtained from a photographed image. As illustrated in Fig. 5, the maximum value in one-dimensional data secured through the first marker unit (110) and / or the second marker unit (120) of the marker module (100) can be obtained, for example, through a mathematical method.

[0087] As an example, the step of determining the displacement of FIG. 9 (step 830) may include the steps of performing a Fast Fourier Transform (FFT) on a one-dimensional pixel value distribution along a reference line, extracting a plurality of valid components from the result of performing the FFT, and formulating the one-dimensional pixel value distribution based on at least one of the magnitude, wavenumber, or phase of the extracted valid components.

[0088] This will be described in more detail with reference to Fig. 5. As shown in Fig. 5, for example, a one-dimensional pixel value distribution secured along a reference line in the x-axis direction may have, for example, a parabolic shape. If FFT transformation is performed on this, analysis of wave components is possible as in Fig. 5. Here, it is possible to extract only meaningful components from the result of FFT transformation. More specifically, for the result of FFT, wave components with significant intensity exceeding a specific reference size (410) can be extracted. Through the extracted components, it is possible to derive a mathematical formula expressing the shape of one-dimensional data. That is, in the result of FFT, the magnitude (A) of each wave component, wave number (f of w n ), phase (φ) information can be obtained, and the formula can be expressed as follows.

[0089]

[0090] y = A1cos (f1x +ϕ1) + A2cos (f2x +ϕ2) + … + A n cos (f n x + φ n )

[0091]

[0092] In this process, the formula y = f(x) can be obtained, and using this, it is possible to track the location of the maximum point using a numerical analysis method.

[0093] That is, according to one aspect, the step of determining the displacement (step 830) may be configured to determine a maximum point for the one-dimensional pixel value distribution using numerical analysis based on a formula for the one-dimensional pixel value distribution. In non-limiting but more specifically, the step of determining the displacement (step 830) may determine a maximum point for the one-dimensional pixel value distribution based on a bisection method, but is not limited thereto.

[0094] According to one aspect, the step of determining the displacement (step 830) may be configured to determine the maximum point for the one-dimensional pixel value distribution by performing an area comparison between two sub-regions divided based on the midpoints of any two points using a formula for the obtained one-dimensional pixel value distribution, and updating the maximum point to the midpoints of both ends of the sub-region having the larger area.

[0095] In relation to this, Fig. 6 describes the procedure of determining the maximum point location based on the bisection method through a formula. As shown in Fig. 6, for example, in the first step (510), any two points (x 01 , x 02 ) and select two points like this (x 01 , x 02 ) of the center (x c = ( x 01 + x 02 ) / 2 ), pixel value distribution curve and x-axis and two points (x 01 , x 02 ) can be divided into areas consisting of straight lines parallel to the y-axis passing through the y-axis. In the second step (520), the areas can be compared based on the center, and the area A1 of the left area and the area A2 of the right area can be compared. Since the formula for the pixel value distribution is secured, the area of ​​each is, x 01 Inland x cThe integral up to and x c from x 02 It can be easily determined by calculating the integral value up to . In the example of Fig. 6, since the area of ​​region A1 is larger, a larger area is selected and the two ends of the region, i.e., x 01 and x c You can select again, x c The location of x 02 can be reset. In the third step (530), the area comparison based on the midpoint is performed again for the two points updated in this way, and the procedure of selecting a wide area again and resetting to both ends including the midpoint can be repeated. Through this repetition, x c The position of can move to the maximum point of the data shape function f(x). At each iteration, the areas A1 and A2 can be derived using integration. Therefore, it is possible to determine the maximum point numerically based on the formula for the one-dimensional pixel value distribution. In this way, when the maximum point for the one-dimensional pixel value distribution is determined, the reference position of the marker part in each frame is determined, so the displacement of the reference position in the target frame with respect to the reference frame can be derived as the displacement of the target object.

[0096]

[0097] FIG. 7 is an exemplary diagram of a marker module having a reflector or a light source according to one embodiment of the present invention. As illustrated in FIG. 7, in the marker module (600) having a reflector or a light source according to one embodiment of the present invention, at least a portion of the first central strip (611) may be configured as a reflector (611r) that reflects light, or at least a portion of the second central strip (621) may be configured as a reflector (621r) that reflects light. Alternatively, at least a portion of the first central strip (611) may be configured as a light source, or at least a portion of the second central strip (621) may be configured as a light source. By configuring at least a portion of the central strips as a light source and / or a reflector in this way, it is possible to enable image analysis-based displacement monitoring of a target object to which the marker module is attached from a distance even at night.

[0098] According to another aspect, in a marker module (600) having a reflector or a light source according to one embodiment of the present invention, at least one of a point reflector or a point light source may be arranged at at least one corner (610a) of the first marker portion (610) and / or at least one corner (620a) of the second marker portion (620). Therefore, in an image captured of the marker portion at night, the position of the marker portion can be easily determined, and the starting point of the reference line can be more accurately determined by the connection between the corners.

[0099]

[0100] FIG. 10 is an exemplary diagram of a structural safety evaluation procedure of a target object according to one aspect of the present invention. As illustrated in FIG. 10, a method for determining object displacement based on image analysis of an object according to one aspect of the present invention may further include a step (step 840) of calculating an evaluation value for structural safety of the target object in the target frame by using the difference between the first direction displacement of the target object determined based on the first marker unit (110) in the target frame and the second direction displacement of the target object determined based on the second marker unit (120). In the case of a given object, even when the allowable limit for the displacement in the first direction and the allowable limit for the displacement in the second direction are not reached, if the difference between the displacement in the first direction and the displacement in the second direction exceeds a predetermined threshold value, the structural safety may not be secured. Accordingly, according to one aspect, in response to a determination that a difference between a first direction displacement of the target object determined based on the first marker portion (110) in the target frame and a second direction displacement of the target object determined based on the second marker portion (120) exceeds a predetermined threshold, it may be determined that the target object is in a structurally unsafe state.

[0101]

[0102] FIG. 8 is a conceptual diagram for detecting distance direction displacement or tilting according to one embodiment of the present invention, and FIG. 11 is an exemplary diagram for a procedure for detecting distance direction displacement and tilting according to one aspect of the present invention.

[0103] As illustrated in FIGS. 8 and 11, a method for determining object displacement based on image analysis of an object according to one aspect of the present invention may further include a step (step 850) of calculating at least one of a distance direction displacement of a target object in a target frame relative to a reference frame, a tilt direction of the target object, or a tilt degree of the target object, based on at least one of a second direction length (111d) of a first central strip (111) in the target frame, or a first direction length (121d) of a second central strip (121).

[0104] According to the steps of the method for determining object displacement based on image analysis of an object according to one aspect of the present invention described above, it is possible to determine when a target object has displacement in a first direction and / or a second direction. According to one aspect of the present invention, it is also possible to determine not only displacement in the first direction and / or the second direction, but also displacement in the direction of the distance between the target object and the image capturing device.

[0105] More specifically, but not necessarily, in response to a determination that at least one of the second direction length (111d) of the first central strip (111) or the first direction length (121d) of the second central strip (121) has decreased, it may be determined that a displacement has occurred in a direction away from the imaging device. Conversely, in response to a determination that at least one of the second direction length (111d) of the first central strip (111) or the first direction length (121d) of the second central strip (121) has increased, it may be determined that a displacement has occurred in a direction toward the imaging device.

[0106] In addition, according to one aspect, in response to a determination that one of the second direction length (111d) of the first center strip (111) and the first direction length (121d) of the second center strip (121) has decreased and the other has increased, it may be determined that tilting has occurred in the target object, and the tilted direction may be determined based on which of the second direction length (111d) of the first center strip (111) and the first direction length (121d) of the second center strip (121) has increased. In addition, it may be determined that the degree of tilting is higher as the difference between the second direction length (111d) of the first center strip (111) and the first direction length (121d) of the second center strip (121) increases.

[0107]

[0108] Referring again to FIG. 2, a marker module (100) attached to a target object for measuring displacement of the target object according to one aspect of the present invention will be described again. The marker module (100) according to one aspect includes a first marker portion (110), and the first marker portion (100) may include a first central strip (111) arranged in a first direction and first background portions (113-1, 113-2) arranged on both sides of the first central strip. The marker module (110) may be used to acquire a photographed image of the first marker portion over a plurality of frames, acquire a one-dimensional pixel value distribution along a first reference line arranged in a second direction orthogonal to the first direction on the first marker portion in a reference frame among the plurality of frames, acquire a one-dimensional pixel value distribution along the first reference line in a target frame among the plurality of frames, and determine a first direction displacement of a target object in the target frame with respect to the reference frame based on a positional difference between a first direction representative value according to the one-dimensional pixel value distribution along the first reference line of the reference frame and a first direction representative value according to the one-dimensional pixel value distribution along the first reference line of the target frame.

[0109]

[0110] FIG. 12 is a block diagram showing an exemplary configuration of a computing system in which methods according to one embodiment of the present invention can be performed.

[0111] Referring to FIG. 12, a computing system (1100) may include flash storage (1110), a processor (1120), a RAM (1130), an input / output device (1140), and a power supply (1150). In addition, the flash storage (1110) may include a memory device (1111) and a memory controller (1112). Meanwhile, although not shown in FIG. 12, the computing system (1100) may further include ports for communicating with a video card, a sound card, a memory card, a USB device, or the like, or for communicating with other electronic devices.

[0112] The computing system (1100) may be implemented as a personal computer or a portable electronic device such as a laptop computer, a mobile phone, a personal digital assistant (PDA), or a camera.

[0113] The processor (1120) can perform specific calculations or tasks. Depending on the embodiment, the processor (1120) can be a microprocessor, a central processing unit (CPU). The processor (1120) can communicate with the RAM (1130), the input / output device (1140), and the flash storage (1110) via a bus (1160), such as an address bus, a control bus, and a data bus.

[0114] According to one embodiment, the processor (1120) may also be connected to an expansion bus, such as a Peripheral Component Interconnect (PCI) bus.

[0115] RAM (1130) can store data required for the operation of the computing system (1100). For example, any type of random access memory including DRAM, mobile DRAM, SRAM, PRAM, FRAM, MRAM, and RRAM can be used as RAM (1130).

[0116] The input / output device (1140) may include input means such as a keyboard, keypad, mouse, etc., and output means such as a printer, display, etc. The power supply (1150) may supply an operating voltage necessary for the operation of the computing system (1100).

[0117]

[0118] The method according to the present invention described above can be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording media that store data that can be deciphered by a computer system. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices. Furthermore, the computer-readable recording medium can be distributed across computer systems connected to a computer communications network, and stored and executed as readable code in a distributed manner.

[0119] Although the present invention has been described with reference to the drawings and embodiments, it does not mean that the scope of protection of the present invention is limited by the drawings or embodiments, and it will be understood that a person skilled in the art can modify and change the present invention in various ways without departing from the spirit and scope of the present invention as described in the following claims.

[0120] Specifically, the described features may be implemented within digital electronic circuitry, or within computer hardware, firmware, or combinations thereof. The features may be implemented, for example, in a computer program product embodied within a storage device within a machine-readable storage device for execution by a programmable processor. And the features may be implemented by a programmable processor executing a program of instructions for performing the functions of the described embodiments by operating on input data and generating output. The described features may be implemented within one or more computer programs executable on a programmable system comprising at least one programmable processor, at least one input device, and at least one output device coupled to receive data and instructions from a data storage system, and to transmit data and instructions to the data storage system. A computer program comprises a set of instructions that can be used directly or indirectly within a computer to perform a particular operation for a given result. A computer program may be written in any programming language, including compiled or interpreted languages, and may be used in any form, including as a module, component, subroutine, or other unit suitable for use in another computing environment, or as a standalone program.

[0121] Suitable processors for executing the program of instructions include, for example, both general-purpose and special-purpose microprocessors, and either a single processor or multiple processors of another type of computer. Also suitable storage devices for implementing the computer program instructions and data implementing the described features include, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic devices such as internal hard disks and removable disks, magneto-optical disks, and all forms of non-volatile memory including CD-ROM and DVD-ROM disks. The processor and memory may be integrated within or added to application-specific integrated circuits (ASICs).

[0122] Although the present invention described above is described based on a series of functional blocks, it is not limited to the above-described embodiments and the attached drawings, and it will be apparent to those skilled in the art to which the present invention pertains that various substitutions, modifications, and changes are possible within a scope that does not depart from the technical spirit of the present invention.

[0123] The combination of the above-described embodiments is not limited to the above-described embodiments, and various combinations may be provided in addition to the above-described embodiments depending on implementation and / or needs.

[0124] In the above-described embodiments, the methods are described based on a flowchart as a series of steps or blocks. However, the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps described above. Furthermore, those skilled in the art will understand that the steps depicted in the flowchart are not exclusive, and other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention.

[0125] The above-described embodiments include examples of various aspects. While not all possible combinations to illustrate the various aspects can be described, those skilled in the art will recognize that other combinations are possible. Accordingly, the present invention is intended to encompass all other alterations, modifications, and variations within the scope of the following claims.

Claims

1. A step of acquiring a photographed image of a first marker portion across a plurality of frames, the first marker portion having a first central strip arranged in a first direction and a first background portion arranged on both sides of the first central strip, and attached to a target object; A step of obtaining a one-dimensional pixel value distribution along a first reference line arranged in a second direction orthogonal to the first direction on the first marker section in a reference frame among the plurality of frames, and obtaining a one-dimensional pixel value distribution along the first reference line in a target frame among the plurality of frames; and A step of determining a first direction displacement of a target object in a target frame with respect to the reference frame based on a positional difference between a first direction representative value according to a one-dimensional pixel value distribution along a first reference line of the reference frame and a first direction representative value according to a one-dimensional pixel value distribution along a first reference line of the target frame; comprising; A method for determining object displacement based on image analysis of the object.

2. In paragraph 1, The step of obtaining the above photographed image is: Further acquiring a photographed image of a second marker portion over a plurality of frames, the second marker portion having a second central strip arranged in the second direction and a second background portion arranged on both sides of the second central strip, and attached to a target object; The step of obtaining the above pixel value distribution is: In the reference frame among the plurality of frames, a one-dimensional pixel value distribution is obtained along a second reference line arranged in a first direction on the second marker section, and in the target frame, a one-dimensional pixel value distribution is further obtained along the second reference line. The step of determining the displacement further determines the second direction displacement of the target object in the target frame with respect to the reference frame based on the positional difference between the second direction representative value according to the one-dimensional pixel value distribution along the second reference line of the reference frame and the second direction representative value according to the one-dimensional pixel value distribution along the second reference line of the target frame. A method for determining object displacement based on image analysis of the object.

3. In paragraph 2, The above first marker part and second marker part, Attached to the measurement points of the target object at a predetermined distance apart from each other, A method for determining object displacement based on image analysis of the object.

4. In paragraph 1, The first direction representative value according to the one-dimensional pixel value distribution according to the first reference line is, The point corresponding to the maximum point of the above one-dimensional pixel value distribution, A method for determining object displacement based on image analysis of the object.

5. In paragraph 1, The step of determining the above displacement is: A step of performing a Fast Fourier Transform (FFT) on the one-dimensional pixel value distribution, extracting a plurality of valid components from the result of performing the FFT, and formulating the one-dimensional pixel value distribution based on at least one of the magnitude, wavenumber, or phase of the extracted valid components. A method for determining object displacement based on image analysis of the object.

6. In paragraph 5, The step of determining the above displacement is: Using numerical analysis based on the formula for the above one-dimensional pixel value distribution, the maximum point for the above one-dimensional pixel value distribution is determined. A method for determining object displacement based on image analysis of the object.

7. In paragraph 6, The step of determining the above displacement is: Using the formula for the above one-dimensional pixel value distribution, the maximum point for the one-dimensional pixel value distribution is determined by performing an area comparison between two sub-regions divided based on the midpoint of two arbitrary points, and updating the maximum point to the midpoint of both ends of the sub-region having a larger area. A method for determining object displacement based on image analysis of the object.

8. In paragraph 6, The step of determining the above displacement is: Determining the maximum point for the above one-dimensional pixel value distribution based on the bisection method, A method for determining object displacement based on image analysis of the object.

9. In paragraph 1, At least a portion of the first central strip, It consists of a reflector that reflects light, At least one of the corners of the first marker portion, At least one point reflector or point light source is placed, A method for determining object displacement based on image analysis of the object.

10. In paragraph 2, A step of calculating an evaluation value for structural safety of the target object in the target frame by using the difference between the first direction displacement of the target object determined based on the first marker portion in the target frame and the second direction displacement of the target object determined based on the second marker portion; further comprising: A method for determining object displacement based on image analysis of the object.

11. In paragraph 2, A step of calculating at least one of a distance direction displacement of the target object in the target frame with respect to the reference frame, a tilted direction of the target object, or a tilted degree of the target object, based on at least one of a second direction length of the first central strip in the target frame, or a first direction length of the second central strip; further comprising; A method for determining object displacement based on image analysis of the object.

12. In paragraph 1, The photographed image of the first marker section above is, Obtained based on cameras mounted on unmanned aerial vehicles, A method for determining object displacement based on image analysis of the object.

13. A marker module attached to a target object to measure the displacement of the target object. The above marker module comprises a first marker section, The above first marker part, a first central strip arranged in the first direction; and A first background portion is provided, which is arranged on both sides of the first central strip; The above marker module, Acquire a photographed image of the first marker portion over a plurality of frames; In a reference frame among the plurality of frames, a one-dimensional pixel value distribution is obtained along a first reference line arranged in a second direction orthogonal to the first direction on the first marker portion, and in a target frame among the plurality of frames, a one-dimensional pixel value distribution is obtained along the first reference line; and Determine a first direction displacement of a target object in a target frame with respect to the reference frame based on a positional difference between a first direction representative value according to a one-dimensional pixel value distribution along a first reference line of the reference frame and a first direction representative value according to a one-dimensional pixel value distribution along a first reference line of the target frame; used for; A marker module attached to a target object to measure the displacement of the target object.

14. A system for determining object displacement based on image analysis of the object, said system comprising: An image capturing unit that captures a photographed image of a first marker portion across a plurality of frames, the first marker portion having a first central strip arranged in a first direction and a first background portion arranged on both sides of the first central strip, and is attached to a target object; and A data processing unit configured to determine displacement of an object based on the above-described photographed image; The above data processing unit, In a reference frame among the plurality of frames, a one-dimensional pixel value distribution is obtained along a first reference line arranged in a second direction orthogonal to the first direction on the first marker portion, and in a target frame among the plurality of frames, a one-dimensional pixel value distribution is obtained along the first reference line; and Determine a first direction displacement of a target object in the target frame with respect to the reference frame based on a positional difference between a first direction representative value according to a one-dimensional pixel value distribution along a first reference line of the reference frame and a first direction representative value according to a one-dimensional pixel value distribution along a first reference line of the target frame; A system for determining object displacement based on image analysis of the object.

Images