A precise distance measuring device that processes mono camera images-cammon-meter

Abstract

Cammon-Meter is an image processing-based distance measuring device that can measure the distance of a specific object to the platform with a 0.5% error margin. The invention consists of a mobile digital camera structure. It also has an electro-mechanical hardware that controls the movement of the mobile digital camera. This hardware also performs the tasks of image processing, data storage, and operation of the developed software and interface. The mobile digital camera first takes a reference image with the initiation of the measurement process. It can then autonomously perform precise linear movement. With the developed image processing software, precise linear movement of the mobile digital camera is provided to detect a pixel width specific to the target object distance. These values are used in the target object distance measurement calculation.

Description

DescriptionTitle of Invention: A Precise Distance Measuring Device That Processes Mono Camera Images - Cammon-Meter Technical Field

[0001] Our invention is an image processing based system consisting of a mono camera and embedded hardware that can measure the distance of a specific object to the platform with a 0.5% error margin.

[0002] Thanks to the invention, the distance of any object (motionless during measurement) determined in the image can be measured very precisely. In addition, the camera only moves linearly in the system. With these features, it differs from existing inventions / patents..

[0003] Invention,

[0004] In precise distance measurement applications;

[0005] 1- The target object is determined manually,

[0006] 2- The camera's initial position is recorded as a reference,

[0007] 3- The camera position calibration is performed autonomously with the developed hardware and software,

[0008] 4- The images taken with the mono camera are processed with image processing techniques,

[0009] 5- The target object distance is calculated with a maximum tolerance of 0.5% with the data obtained as a result of image processing,

[0010] It concerns the "Precision Distance Measuring Device Processing Mono Camera Images" with electromechanical, autonomous position calibration, with which all target object distances can be measured.Patent Literature

[0011] When patent and utility model applications made to international (World Intellectual Property Organization / patent scope) patent offices are searched with different keywords, different results are obtained. For example, there are 383423 results with “distance measure”, 10814 results with “mono camera”, 80205 results with “distance measure with camera” and 4590 results with “mono camera distance measure”. However, keyword-based searches alone are not sufficient and may also include patents that are not compatible with the subject of the invention. Nevertheless, the numerical results in the search results indicate that many patents have been obtained for image processing -based distance measurement systems. Among the patents registered with the Turkish Patent Office, 36 patents were found with the search term“distance measure”, 5 with “stereo camera”, 1 with “mono camera”, 0 with “distance measure with camera” and 6875 with “distance”.

[0012] When these patents were filtered, 1 patent was found that contained similarity to our invention (2021 / 016384). It was determined that the other patents were not completely similar. Patent number 2021 / 016384 was obtained by the same inventors.

[0013] The similarities and differences between the patent number 2021 / 016384 and the invention in question are given in the table 2 below.

[0014] Table 1: Comparison of Patent No. 2021 / 016384 and the Invention Subject to Application

[0015] The patent (2021 / 016384) compared in Table 1 uses the Improved Triangulation Method as a method. A fixed and a moving camera are used to use this method. The fixed camera image is taken as a reference, the moving camera moves angularly and linearly to find the optimum angle and distance between the lenses required for the measurement. The distance measurement is calculated with the optimum angle and distance between the lenses values. There is a single camera in the invention in question. This camera moves only linearly and the distance of the target object is measured with the principle of finding the optimum pixel width of the object whose distance will be measured.

[0016] Information on other patents examined is given in Table 2.

[0017] There is no image processing-based distance measuring device in the industrial market. Existing distance measuring devices are RADAR, LIDAR, LASER based distance measuring devices. The device in question is an alternative to existing measuring devices and is measured by image processing. When literature and patents are examined, it is seen that there are many studies on image processing-based distance measurement and that patents have been obtained using different image processing techniques than those used in the study. However, it has been observed that thereare deficiencies in the implementation of the patents obtained and the development of a distance measuring device. Despite the patents being obtained, the fact that the device has not been produced and launched on the market supports this prediction. It is thought that the invention we plan to develop will largely eliminate this deficiency in the market. In addition, although both Patent No. 2021 / 016384 and the invention in question can perform precise distance determination. It is seen that the error rate and cost of the invention in question are advantageous compared to the product in question No. 2021 / 016384 due to its higher sensitivity and single axis movement (linear).

[0018] Table 2:Patent List

[0019] It was observed that the measurement distance and sensitivity rates were not given in the vast majority of the patents that were scanned. Although it is not important to provide such data in patents, inferences made by examining the methods subject to the patent during the literature review phase, in general, no patent content was found that could obtain sensitive results at long distances.

[0020] In the patent content given as number 11 in Table 2, it is said that it can measure in mm at distances more than 100m. In this patent content, image sensors to be mounted side by side on a single chassis and the processing of images taken from these sensors are suggested. Although the patent was obtained as of 2010, no device using this method has been found in the market. In addition, using a large number of cameras andprocessing all these images together will increase the processing load. Other patents using array cameras are (12,13,15)*.

[0021] Patents numbered 8 and 9 are methods performed with a mono camera. In these, the method was developed based on either the camera being mobile (8)1or the object being mobile (9)*. Thus, separate images are taken when the same object is at different distances. In addition, when the speed of movement and the time between two images are known, the distance can be determined.

[0022] Generally, the patented methods perform pixel-based distance determination (1,2,3,4,6,10,14)*. Additional, in some of them, angular and linear movements are mentioned (2,6,14,16)*, and these processes are used one by one. In these studies, especially those using linear movement, the movement is done manually (2)* and is not used as an active parameter in the distance calculation. In our invention, the pixel width value of the target object will be found as a result of the linear movement and will be used as an active parameter. For this reason, it is important to determine the pixel width precisely.

[0023] In addition, in some patents, light is refracted by placing prisms in front of the cameras (5)*. The effect of this method on calculations has not been sufficiently explained. For this reason, it is difficult to make inferences about the success of the method. In some, depth mapping is aimed.Brief Description of Drawings

[0024] Features, aspects and advantages of the present invention will become better understand when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings: Fig.l

[0025] It is a Cammon-Meter front viewFig.2

[0026] It is a Cammon-Meter rear viewFig.3

[0027] It is a Power unit block diagram [1.9]Fig.4

[0028] It is a Control unit block diagram [1.11]Fig.5

[0029] It is a Operator Panel block diagram [1.10]

[0030] Each part in the figures prepared for a better understanding of this invention is numbered separately. The explanation of each numbered part is as follows;

[0031] 1.1 Digital camera

[0032] 1.2 Balance unit

[0033] 1.3 Upper slide

[0034] 1.4 Gear Shaft

[0035] 1.5 Lower slide

[0036] 1.6 Linear motion motor

[0037] 1.7 Camera motion module lower support

[0038] 1.8 Camera motion module upper support

[0039] 1.9 Power unit

[0040] 1.10 Operator panel

[0041] 1.11 Control panel

[0042] 1.12 Balance sensor physical indicator

[0043] 1.13 Power unit charge input socket

[0044] 1.14 Power unit 1 st battery

[0045] 1.15 Power unit 2nd battery

[0046] 1.16 5V voltage regulation card

[0047] 1.17 12V voltage regulation card

[0048] 1.18 24V voltage regulation card

[0049] 1.19 Power unit GND and +24V output socket

[0050] 1.20 Power unit GND and +12V output socket

[0051] 1.21 Power unit GND and +5V output socket

[0052] 1.22 Linear motor control card supply voltage input

[0053] 1.23 Main-control card supply voltage input

[0054] 1.24 External motor control card supply voltage input

[0055] 1.25 Linear motor control card

[0056] 1.26 Main control card

[0057] 1.27 Balance control card

[0058] 1.28 Balance unit input output socket

[0059] 1.29 Main control card input output socket

[0060] 1.30 Linear motor input output socket

[0061] 1.31 Mini PC

[0062] 1.32 Mini PC touch screen input / output socket

[0063] 1.33 Mini PC external input / output pins

[0064] 1.34 Mini PC external input / output component pins

[0065] 1.35 Mini PC USB input / output sockets

[0066] 1.36 Mini PC HDMI output socket

[0067] 1.37 Mini PC Ethernet input / output socket

[0068] 1.38 Mini PC audio output socket

[0069] 1.39 Mini PC mini USB input / output pins

[0070] 1.40 Mini PC external memory card socket

[0071] 1.41 Mini PC processor

[0072] 1.42 Touch screen

[0073] 1.43 Touch screen input / output socket

[0074] Explanation of Parts Shown in Pictures with Reference Numbers

[0075] The front face view of the Cammon Meter is given in [Fig.l], the back face view and control block are given in [Fig.2], the block diagram of the power unit is given in [Fig.3], the block diagram of the control card is given in [Fig.4] and the block diagram of the operator panel is given in [Fig.5].

[0076] The Cammon Meter front view [Fig.l] includes the digital camera [1.1], balance unit [1.2], ball screw shaft [1.4] for linear movement, support bearing upper slide [1.3], support bearing lower slide [1.5], linear motor [1.6] that provides the linear movement of the camera (1.1), motion module lower support [1.7] that allows the digital camera[1.1] to be fixed to the main table, and the motion module upper support [1.8] of the digital camera [1.1].

[0077] Cammon Meter rear view [Fig.2], there is a digital camera [1.1], balance unit [1.2], power unit [1.9], operator panel [1.10], control panel [1.11], balance sensor physical indicator [1.12], there is a power unit [1.9] with + / -5V, + / -12V and + / -24V input and output sockets on it, an operator panel [1.10] where the Cammon Meter allows the user to take the image and select the target object, where the data is recorded and the software is operated, a control unit [1.11] with a start button on it that allows the necessary reference values to be entered in the Cammon Meter system, a balance unit[1.2] that provides the inclination of the system with the ground and a physical balance sensor physical indicator [1.12].

[0078] In the power unit block diagram [Fig.3]; there are power unit charge input socket [1.13], battery group inside the power unit [1.14 and 1.15], 5V voltage regulator card [1.16], 12V voltage regulator card [1.17], 24V voltage regulator card [1.18], Power unit GND and +5V output socket [1.19], Power unit GND and +12V output socket [1.20], Power unit GND and +24V output socket [1.21]. In the control unit block diagram [Fig.4]; there are the necessary supply voltage input socket for the linear motor control card [1.22], the supply voltage input socket for the main control card[1.23], the necessary supply voltage input socket for the external motor control card[1.24], the linear motor control card [1.25], the main control card [1.26], the balance control card [1.27], the balance unit connection socket [1.28], the input output socket for the main control card and the camera and interface screen [1.29], and the linear motor control card input output socket [1.30]. The operator panel block diagram [Fig.5] consists of two parts: Mini PC [1.31] and touch screen [1.42]. On the Mini PC [1.31]; there are touch screen input / output

[0079] socket [1.32], external input output pins [1.33], external component input output pins [1.34], USB input output socket [1.35], HDMI output socket [1.36], Ethernet input output socket [1.37], audio output socket [1.38], micro USB input output socket [1.39], external memory card socket [1.40], central processor unit [1.41]. On the touch screen [1.42]; There is a touch screen input / output socket [1.43]. Description of Embodiments

[0080] Technical Problems That the Invention Aims to Solve

[0081] Since the invention aims at image processing based distance determination, the technical problems detected for image processing based distance determination studies have been examined.

[0082] In order to extract the depth of an object in a stereo image, the object must be imaged from two different points. If the displacement in the image taken from two different points and the distance between these two points are known, a calculation can be made about the depth of that object. In the triangulation method used in the literature, it is seen that high camera resolutions are important in determining the distance.

[0083] In image processing-based stereo camera distance detection studies, the distance of the object is calculated using the difference in the horizontal coordinates of the images of the object taken by the cameras on the horizontal axis (shift amount - Ax) and the actual distance between the positions of the cameras. The shift amount of Ax depends on the difference in the horizontal axis of the cameras (1), the distance between the object and the cameras (Z) and the resolution of the cameras. For example, if we assume that the horizontal resolution of the cameras is 1 pixel, Ax will always be 0 regardless of the distances. In this case, there will be no possibility of distance detection. While increasing the horizontal resolution and 1 distance of the camera causes Ax to increase, increasing the Z distance causes Ax to decrease. Increasing the camera resolution loses its meaning after a certain value due to cost and processing load. Increasing the 1 distance negatively affects the ease of use of the device. As a result, optimum solutions cannot be produced.Examples

[0084] PPM (Pixel Per Meter) number is the most basic and important value affecting Ax. The second value that affects Ax is the horizontal width (FoV) value of the area you want to display. Since the FoV value will increase as the distance of the object or area to be imaged from the camera increases, the PPM value will decrease as the distance increases. In this case, it reduces the image quality and Ax value.

[0085] In a fixed focal length camera, the farther the target object is, the wider the FoV area of the camera. Increasing the FoV width decreases the PPM value. This increases the pixel width on the distant object and reduces the image quality. The equation 1 below shows that the FoV area changes depending on the viewing angle “a” and the distanceof the target “Z”. In equation 2, the PPM value and in equation 3, the amount of image shift “Ax” on the camera's image screen are calculated in pixels.

[0086] FOV = tan ( f ) *Z*2

[0087] [Math.l]

[0089] In the equations;

[0090] a, camera view angle (degrees)

[0091] Z, distance of target object from platform (meters)

[0092] Wpx, horizontal resolution of the camera (pixels)

[0093] L, distance between lenses(meters)

[0094] FoV, camera's viewing width for a given distance (meters)

[0095] PPM, number of pixels per 1 meter (pixels)

[0096] However, even if a high-resolution camera is used, it is clear that as the distance increases, the object to be detected will occupy less space on the screen (for a certain size object). For this reason, it is clear that at long distances, there will be no pixel differences between the object images formed on the image screens of stereo cameras with parallel image acquisition axes, and it will not be possible to calculate the distance. In examples provides 4 separate examples that calculate the distance of the object to the cameras depending on the amount of Ax shift with stereo cameras. The calculations were made for a high-resolution (4608x3456 pixels) parallel stereo camera with a fixed 1 distance.

[0097] Examples of variation of Ax value with respect to Z and I for long distances

[0098] Table 3 Examples of variation of Ax value with respect to Z and I for long distances

[0099] In Exemples ., although a high-resolution stereo camera (4K-4608X3456 pixels) is used, the pixel difference (Ax) in the horizontal plane formed on the image screens of the target object does not change for 600, 800, 900 and 1000 meters. This means that the distance measuring device gives the same result for all 4 distance values, and it is clear that the measurement precision is very low and cannot be accepted.

[0100] There are image processing based distance detection studies in the literature using stereo and mono cameras. However, most of these studies could not obtain results that could measure with an error margin below 4%. There are two main problems in studies conducted with mono cameras. The first of these is the need to use a target board or to have information about the object dimensions. Because these methods make inferences based on the area covered by the object in the image. The second main problem is that the change in the area covered by the object as the distance increases is expressed with very small values or even there is no change after a certain distance. The reason for this is that the PPM value decreases as the distance increases.

[0101] In studies conducted with stereo cameras, distance determination is based on finding the shift in the horizontal (X) axis in pixels. Similar to the second problem in mono cameras, the shift amount decreases as the distance increases in stereo cameras, and even reaches 0 pixels (Ax) after a certain distance, depending on the resolution and the distance between the camera lenses (see examples.

[0102] In Table 4 shows FoV at different object distances, pixel width at different distances, PPM, Ax, pixel difference and max error values. As can be seen in Table 4, as the distance increases, the decrease in the PPM value causes the maximum error to increase.

[0103] Table 4 Effect of distance increase on Ax change and maximum error in distance detection studies depending on Ax.

[0104] When the values given in Table 4. are examined, it is seen that while the measurement can be made with an average resolution of 1 / 8 = 0.125m at 1 Im, the measurement resolution drops to an average of 0.2m at 12m, 0.615m at 20m, and 0.769m at 30m and a non-linear graph is formed depending on the distance, and resolution cannot be obtained at high increases in distance.

[0105] In the literature, only one study was found using a stereo camera for precise distance determination. A patent was obtained from TPE for this study (2021 / 016384). In this study, the precise determination of “|3” angle and lgvalues in stereo cameras and image processing-based distance measurement technology minimizes pixel dependency (Ax) and increases measurement precision. Thus, the improved triangulation method was used.

[0106] The subject of the invention is that there is only linear movement in the precise distance determination made with the Mono camera. With this movement, the real width of a pixel belonging to the target object is found. This width depends on the distance of the object to the camera. The precise determination of the pixel width also ensures that the distance is found precisely. It is a completely different method from the triangulation method and other measurement methods with mono cameras. Thepurpose of the invention is to precisely find the target object distance using the pixel width determination method.Distance Measurement Steps

[0107] The invention performs the measurement process by performing the following steps.

[0108] 1- The device is charged via the power unit charging input socket [1.13].

[0109] 2- The measuring device is placed on a stable surface. The device is powered via the power unit [1.9] using the power unit 1. Battery [1.14], power unit 2. Battery [1.15], 5V voltage regulation card [[1.16], 12V voltage regulation card [[1.17], 24V voltage regulation card [[1.18], Power unit GND and +24V output socket [1.19], Power unit GND and +12V output socket [1.20], Power unit GND and +5V output socket [1.21], Linear motor control card supply voltage input [1.22], Main-control card supply voltage input [1.23], external motor control card supply voltage input [1.24].

[0110] 3- The device's inclination is set to 0 degrees manually / autonomously with the help of the balance unit [1.2]

[0111] 4- The operator can control the camera movement module lower support [1.7] and camera movement module upper support via the interface control touch screen [1.42] The digital camera [1.1] fixed with [1.8] determines the target object in its image, marks it with a pointer and starts the measurement process via the control unit [1.11].

[0112] 5- The digital camera [1.1] image is processed on the mini PC [1.31]. The data obtained on the mini PC is transferred to the main control card [1.26] and the balance unit [1.2] is controlled. The linear motion motor [1.6] controls the linear motion of the digital camera [1.1] via the gear shaft [1.4] mounted between the upper slide [1.3] and the lower slide [1.5] and the camera motion module lower support [1.7] - camera motion module upper support [1.8]. The linear motion motor [1.6] works synchronously with the mini PC processor [1.41] while controlling the linear motion of the digital camera [1.1] (The methods used and the electromechanical process flow are given in detail below).

[0113] 6- The object distance is calculated on the mini PC [1.31].

[0114] The calculation gives the width of the pixel at a certain distance. The distance information is found by providing the necessary conditions and mathematically calculating this information..

[0115] Linear motion control and pixel width calculation with image processing

[0116] The reference object image taken from the reference position with a digital camera and marked with a pointer is processed with the developed image processing software to determine the features of the object (object segmentation, location of the object in the image, etc.). While the digital camera moves on the horizontal axis, images are taken from different positions to determine the object location in this image and compared with the object location in the reference image. This process continues untilsufficient data is reached to find the real width of a pixel belonging to the object in the image.

[0117] Precise position data of the digital camera's horizontal axis movement is recorded by matching it with images.

[0118] In images processed with image processing methods, the linear motion magnitude that allows the object image to shift by 1 pixel is precisely detected and the object distance is calculated with the following equations. pixel width

[0121] In the equations;

[0122] a: Camera view angle(degrees)

[0123] Z: Distance of target object from platform (meters)

[0124] Wpx : Horizontal resolution of the camera (pixels)

[0125] FoV : Camera's viewing width for a given distance (meters)

[0126] The invention focuses on measuring the precise distance of a single object.

[0127] In this context, the purpose of the invention is to design a mono camera-based embedded system that can measure the distance of a visible object at a certain distance to the cameras with a 0.5% error margin.

[0128] For this purpose, the invention will have:

[0129] • 1 high-resolution, optical zoom camera (capable of receiving images and transmitting images via Wi-Fi),

[0130] • A linear actuator for the precise adjustment of the distance between camera locations and the transfer of this parameter to the digital environment,

[0131] • An operator panel that will transfer the transferred images and the obtained results to the user, process them and produce results, and operate the interface (GUI) to be developed for the use of the device,

[0132] • A software that can be controlled via the interface where the user can start the measurement process by making the initial adjustments and receive the results,

[0133] • And a tripod that will allow the device to receive images stably.

[0134] The invention;

[0135] • It can perform object segmentation using image processing techniques,

[0136] • It can superimpose images taken from different locations from a moving camera and make inferences,

[0137] • It can control linear motor drivers using the data it obtains,

[0138] • It can process the equation created using the determined parameters,

[0139] • It can detect the distance of a certain object of visible size to the camera with a tolerance of 0.5%,

[0140] • It can be used in open and closed environments.

[0141] How the invention is applied to industry

[0142] The invention can be used in many areas that require precise distance measurement as a distance measuring device. Examples of areas of use include defense industry systems, construction, mapping, etc. sectors.

[0143] The distance measuring device in question.

[0144] 1- It can be portable within the framework of the electromechanical hardware features used in its design and can be mounted on any mobile vehicle. It is also suitable for use in open and closed environments.

[0145] 2- It has open-source code within the framework of the software features used in the design and does not require a license. Since the image processing techniques and other algorithms implemented in the software framework will be realized on the electronic hardware of the device, it has the feature of working offline. It is not dependent on the internet during the measurement phase. However, it can be connected to the internet when the data needs to be transferred to another location.

Claims

Claims

1. A precise distance measuring device that processes mono camera images Cammon-Meter, comprising: The device has an image processing-based distance measuring device box called "Cammon-Meter" which calculates the distance of the determined target object by using image processing methods after the device operator sets up the device, determines the target object to be measured and starts the distance measurement process.

2. The precise distance measuring device that processes mono camera images Cammon-Meter of claim 1, it contains a movable digital camera (1.1) positioned on an electromechanical system capable of linear movement.

3. The precise distance measuring device that processes mono camera images Cammon-Meter of claim 1 and claim 2, it uses the digital camera (1.1) image with the determined initial position as a reference and moves the camera to different positions, takes images again from these positions, detects the object location in the new image, compares this location with the object location in the reference image and calculates the pixel width of the target object using the obtained data.

4. The precise distance measuring device that processes mono camera images Cammon-Meter of claim 1 and claim 3, it calculates the target object distance by using the pixel width of the target object in the image.

5. The precise distance measuring device that processes mono camera images Cammon-Meter of claim 1, it has an operator panel (1.10) and mini PC (1.31) that autonomously controls the electromechanical system that contains the power unit (1.9), control unit (1.11), operator panel (1.10), balance unit (1.2), balance sensor physical indicator (1.12), reprocesses the data as a result of each mechanical movement, analyzes the data obtained, terminates the mechanical movement when the appropriate data is obtained, calculates the target object distance and transmits it to the user.

Images