Image processing method for processing images from multiple cameras of a ship cleaning robot into one image, computer readable recording medium, computer program, and robot control method using the same

Abstract

The present invention relates to an image processing method for processing a plurality of images into one image, the image processing method including the steps of acquiring at least a left image, a center image, and a right image from a plurality of cameras arranged in a row, generating the left image, the center image, and the right image as a top-view left image, a top-view center image, and a top-view right image, respectively, synthesizing the top-view left image, the top-view center image, and the top-view right image to generate one wide top-view image, and generating and outputting the wide top-view image as a final wide image, thereby reducing fatigue of a robot operator by synthesizing the plurality of camera images into one image.

Description

[Technical field]

[0001] The present invention relates to an image processing method for a ship cleaning robot, and more particularly to an image processing method for processing images from a plurality of cameras of a ship cleaning robot into one image. [Background technology]

[0002] Even for newly constructed ships, the moment they come into contact with seawater, various marine organisms begin to attach to the surface of the underside of the hull (hereinafter referred to as the hull surface).Various marine organisms, from slime to seaweed, barnacles, and mussels, attach to the hull surface, and as the amount of organisms that attach to it decreases the ship's travel speed and increases oil costs by 10 to 30%, so cleaning the hull surface is essential.

[0003] Antifouling paint is applied to the surface of a ship to prevent marine organisms from attaching to it, but this only delays the time it takes for the organisms to attach, and after about a year, organisms such as barnacles and mussels (hereafter referred to as barnacles), which greatly increase water resistance, begin to form colonies and attach to the surface of the ship. In order to remove such organisms, the surface of the ship must be cleaned using divers or ship cleaning robots (hereafter referred to as robots).

[0004] When sunlight or lighting shines into seawater, the light is scattered by minute debris and plankton in the seawater, shortening the recognition distance of the naked eye and the camera. The recognition distance in seawater in ports where ship cleaning is carried out is mostly 20 to 50 cm. Therefore, when using a robot to clean the surface of a ship, it is necessary to consider situations where the recognition distance is about 20 to 50 cm. For example, if a single camera using a wide-angle or fisheye lens is installed in the center of the front of the robot, the recognition distance is usually about 40 to 100 cm from the left to right side of the camera. For example, if the width of the robot is 100 cm and the recognition distance is 50 cm, the entire robot can be seen from the left to right side with one camera, but with a recognition distance of 20 cm, only a 40 cm range from the left to right side of the camera can be seen, so only the center part of the robot can be seen and the left and right sides cannot be seen.

[0005] When cleaning the surface of a ship, there is a clear boundary between the cleaned and uncleaned areas (hereinafter referred to as the cleaning boundary). The robot moves along this cleaning boundary by placing the left or right edge of the cleaning brush attached to the robot, but if the identification distance is 20 cm, the cleaning boundary must be placed near the camera in the center of the robot, which creates a problem that the cleaning time takes more than twice as long as conventional methods. To solve this problem, multiple cameras can be placed at intervals of 20 to 40 cm, but the robot operator must operate the robot while alternately looking at multiple cameras, which increases the operator's mental fatigue.

[0006] For this reason, in order to reduce fatigue in robot operators who operate for long periods of time, technology is needed to combine images from multiple cameras into a single image. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] KR10-1444387B1 [Patent Document 2] KR10-2188721B1 Summary of the Invention [Problem to be solved by the invention]

[0008] SUMMARY OF THE PRESENT EMBODIMENT It is therefore an object of the present invention to provide an image processing method capable of reducing fatigue of a robot operator by combining images from a plurality of cameras into a single image.

[0009] Another object of the present invention is to provide an image processing method capable of reducing fatigue of a robot operator by converting a dark part of an image into a bright part.

[0010] The problems to be solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0011] The object of the present invention is to provide a method for the prevention of the above-mentioned problems, comprising the steps of: This is achieved by an image processing method that processes a plurality of images into one image, the image processing method including the steps of: acquiring at least a left image, a center image, and a right image from a plurality of cameras arranged in a row; generating the left image, the center image, and the right image as a top-view left image, a top-view center image, and a top-view right image, respectively; synthesizing the top-view left image, the top-view center image, and the top-view right image to generate one wide top-view image; and generating and outputting the wide top-view image as a final wide image.

[0012] In this case, it is preferable that each of the left image, the central image, the right image, and the final wide image is a perspective view image having one vanishing point, and each of the top view left image, the top view central image, the top view right image, and the wide top view image is a top view image as seen from above downwards with no vanishing point.

[0013] In addition, the step of generating the left image, the center image, and the right image as a top-view left image, a top-view center image, and a top-view right image, respectively, and the step of generating and outputting the wide top-view image as a final wide image use perspective transformation. If the perspective transformation uses a transformation matrix, the transformation matrix used in the step of generating the left image, the center image, and the right image as a top-view left image, a top-view center image, and a top-view right image, respectively, is an inverse matrix of the transformation matrix used in the step of generating and outputting the wide top-view image as a final wide image.

[0014] Furthermore, it is preferable that the step of synthesizing the top-view left image, the top-view center image, and the top-view right image to generate one wide top-view image includes the step of processing boundary areas in areas where at least two of the top-view left image, the top-view center image, and the top-view right image overlap, as semi-transparent, and removing the remaining areas.

[0015] Furthermore, the step of acquiring at least a left image, a center image, and a right image from the multiple cameras arranged in a row may further include the steps of acquiring brightness information, saturation information, and hue information for each of the left image, the center image, and the right image, inverting the acquired brightness information to acquire inverted brightness information, acquiring first correction information for correcting the brightness information based on the inverted brightness information, applying the first correction information to the brightness information to generate converted brightness information, and generating an image corrected with the converted brightness information, the saturation information, and the hue information.

[0016] In addition, the present invention further includes a step of obtaining second correction information for correcting saturation information based on the inverted lightness information, and a step of applying the second correction information to the saturation information to generate converted saturation information, and the saturation information used in the step of generating an image corrected with the converted lightness information, the saturation information, and the hue information can be replaced with the converted saturation information.

[0017] Furthermore, it is preferable that the step of acquiring first correction information for correcting the brightness information or the step of acquiring second correction information for correcting the saturation information based on the inverted brightness information acquires, for each pixel constituting the final wide image, the first correction information for correcting the brightness information or the second correction information for correcting the saturation information, which is not 0, only when the inverted brightness information is equal to or greater than a threshold value.

[0018] In this case, the first correction information and the second correction information may be the same or different.

[0019] The above object can also be achieved by the second aspect of the present invention, This can be achieved by a computer-readable recording medium having a program for carrying out the above-mentioned method recorded thereon.

[0020] Furthermore, the above object is also achieved by a third aspect of the present invention, This is accomplished by a computer program stored on a medium for performing the above method in combination with hardware.

[0021] Furthermore, the above object is also achieved by a fourth aspect of the present invention, This is achieved by a robot control method including the steps of: constructing a control model for adjusting the moving speed of the robot and the rotation speed of the cleaning brush by learning any one of the images acquired or generated by the image processing method; and adjusting the moving speed of the robot and the rotation speed of the cleaning brush by using the constructed control model based on a final wide image generated by processing images acquired from a plurality of cameras arranged on the robot by the image processing method.

[0022] In this case, the step of adjusting the moving speed of the robot and the rotation speed of the cleaning brush may include a step of displaying to the operator of the robot that the moving speed of the robot and the rotation speed of the cleaning brush are being adjusted, and such display may include using at least one of text, graphics, vibration, sound, and lighting. Effect of the Invention

[0023] According to the image processing method of the present invention, which processes a plurality of images into one image, it is possible to reduce the fatigue of a robot operator by combining a plurality of camera images into one image.

[0024] In addition, the image processing method of the present invention for processing a plurality of images into one image has an advantage that the dark parts of the image can be brightened to reduce fatigue of the robot operator.

[0025] In addition, the robot control method using the image processing method of the present invention has an advantage that the robot operator can clean the ship more neatly while reducing fatigue. [Brief description of the drawings]

[0026] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or restricted by the exemplary embodiments. The same reference numerals in each drawing indicate elements that perform substantially the same functions. [Figure 1] 2 is a flowchart of a video processing method according to the present invention. [Diagram 2] 1 is a diagram showing an embodiment of a video processing method according to the present invention; [Diagram 3] FIG. 1 is a diagram illustrating an example of a video processing method according to the related art. [Figure 4] FIG. 4 is a diagram showing information generated by the video processing method according to the present invention. [Diagram 5]FIG. 2 is a diagram showing an example of an image processed by the image processing method according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Terms including ordinal numbers such as first, second, etc. are used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and the second component may be named the first component, without departing from the scope of the present invention. The terms used in this application are merely used to describe specific embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0028] The terms used in the present invention are selected as general terms currently widely used as much as possible while taking into consideration the functions in the present invention, but these may change depending on the intentions or precedents of the engineers engaged in the field, the emergence of new technologies, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meanings thereof are described in detail in the description of the relevant invention. Therefore, the terms used in the present invention should be defined based on the meanings of the terms and the overall content of the present invention, rather than simply the names of the terms.

[0029] Throughout the specification, when a part is said to "comprise" a certain element, this does not mean to exclude other elements but may further include other elements, unless specifically stated to the contrary.

[0030] 1 is a flow chart of an image processing method according to the present invention. Referring to the drawing, the image processing method according to the present invention essentially includes the steps of acquiring a plurality of camera images (S100), acquiring corresponding top-view images (S200), combining the images to generate one wide top-view image (S300), and finally outputting a final wide image (S400).

[0031] This will be explained in more detail with reference to FIG. 2, which illustrates an embodiment of a video processing method according to the present invention.

[0032] Referring to the drawings, the step of acquiring a plurality of camera images (S100) of the present invention includes acquiring images (hereinafter, referred to as camera images) captured by a plurality of cameras arranged in a line.

[0033] At this time, the camera image is a perspective-view image with one vanishing point, and the images captured using each camera can be called a left image, a center image, and a right image according to their positions. This is named based on the use of three cameras in this embodiment, and if five cameras are used, they can be named as left 1 image, left 2 image, center image, right 1 image, right 2 image, etc., and is not necessarily limited thereto.

[0034] Also, the step of obtaining a corresponding top-view image (S200) of the present invention includes generating the left, center, and right images as a top-view left image, a top-view center image, and a top-view right image, respectively.

[0035] At this time, the top-view left image, top-view center image, and top-view right image are top-view images without a vanishing point, as if the floor is viewed from above. To generate a top-view image from a perspective view image, a perspective transformation is used. The principle of perspective is that distant objects appear small and close objects appear large. A method of transforming an image by applying the principle of perspective is called a perspective transform. In general, in a perspective transform, a transformation matrix is ​​found using only four points in an image to transform the image, and by using this, a planar image like a scan can be created. This perspective transform method is used in lane view image processing, which is used to detect driving lines in autonomous vehicles, and is a universal technology used for various other purposes, so a detailed description will be omitted here.

[0036] Next, in the step of generating one wide top-view image (S300), the top-view left image, the top-view center image, and the top-view right image are combined to generate one wide top-view image. For example, in an area where at least two images overlap, such as the top-view left image and the top-view center image, or the top-view center image and the top-view right image, only the boundary area is left, and all remaining overlapping areas are removed, and then the images are combined.

[0037] In this case, the combination of images may be performed, for example, by processing the boundary area in an area where at least two or more images overlap semi-transparently and removing the remaining area from the image, or by leaving the boundary area in an area where at least two or more images overlap only in one image and removing the remaining area from the remaining image. Processing semi-transparently includes reducing the image information, such as hue information, saturation information, brightness information, etc., of the original images (top view left image, top view center image, and top view right image) by half so that the overlapping boundary areas can be simply added later. Of course, it is also possible to add the overlapping boundary areas by applying different weights to each of the images including the overlapping boundary areas, or to set the overlapping boundary areas so that the parts to be left or removed do not overlap each other. In this drawing, the parts corresponding to the boundary areas in the area where at least two or more images overlap are expressed semi-transparently.

[0038] Finally, the step of outputting a final wide image (S400) includes the step of generating one wide top-view image, which is generated by the synthesis using the perspective transformation, as a final wide image.

[0039] In this case, if the perspective transformation uses a transformation matrix (A) in the step (S200) of acquiring a corresponding top view image from a perspective view image, it is preferable that the transformation matrix used in this step (S400) of acquiring a corresponding final wide image from a wide top view image is the inverse matrix of the previous transformation matrix (A).

[0040] As a result, if the technology of synthesizing multiple camera images with different vanishing points into one based on the image processing method of the present invention is applied to, for example, a ship cleaning robot, it can significantly reduce the fatigue of the robot operator.

[0041] On the other hand, ship cleaning robots can operate either manually or autonomously; when operating manually, a person on land or on the water will operate the robot while watching the camera footage of the robot on a monitor.

[0042] When a robot travels under a large ship, sunlight does not reach it even in the middle of the day, so it must use the lights installed on the robot to travel, but even with those, the light is scattered by tiny objects in the seawater, such as garbage and plankton, and is absorbed by the seawater, so it does not reach far.Even if the camera footage is taken using an ultra-low-light camera that can detect even faint light, unlike ordinary cameras, if the amount of light from the lights installed on the robot that reflects off the surface of the ship and returns to the camera is small, it appears dark on the monitor, and it gets darker rapidly the further away it is from the robot.

[0043] Although the saturation of the camera image can be increased to make even dark areas discernible, simply increasing the saturation may make the colors too dark and cause thin shapes to disappear, making it more difficult to discern objects. Such problems with the conventional technology will be described with reference to FIG. 3, which is a diagram illustrating an example of an image processing method according to the conventional technology. FIG. 3(a) is an original image taken by a camera of a ship cleaning robot while cleaning the surface of an actual ship, and includes a dark area 30a and a fine area 40a including thin shapes. FIG. 3(b) is an image in which both the brightness information and the saturation information have been increased in the original image, FIG. 3(c) is an image in which only the brightness information has been increased in the original image, and FIG. 3(d) is an image in which only the saturation information has been increased in the original image. When the brightness is increased to brighten the dark areas as in FIG. 3(c), the image becomes blurred overall, and when the saturation is increased to clarify the blurred areas as in FIG. 3(d), shape distortion occurs in which thin shapes disappear. Therefore, if the brightness and saturation are both increased to brighten the dark areas while clarifying the blurred areas, the dark areas 30b will be brightened as shown in FIG. 3(b), but other areas will also be brighter than necessary, causing distortion and the fine shapes in the fine areas 40b to disappear.

[0044] Normally, robot operators have to concentrate on looking at a monitor screen for long periods of time, which causes high levels of mental and physical fatigue.In addition, because the distance the robot moves per second is longer than the recognition distance, the operator must spend most of the time looking at the dark areas of the camera image to determine whether there are obstacles or biological fouling on the ship's surface, which causes high levels of fatigue for the operator.

[0045] Therefore, in order to reduce the fatigue of robot operators who operate for long periods of time, a technology that makes it easy to distinguish dark areas in camera images is needed. At this time, such a technology that makes it easy to distinguish dark areas can be used separately from the above-mentioned technology of combining images into one image, or can be used in conjunction with it.

[0046] FIG. 4 is a diagram showing information generated by the video processing method according to the present invention, and FIG. 5 is a diagram showing an example of a video processed by the video processing method according to the present invention.

[0047] Referring to FIG. 4, the image processing method according to the present invention includes a step of acquiring brightness information, saturation information, and hue information based on an original image (S500), a step of inverting the acquired brightness information to acquire inverted brightness information (S600), a step of acquiring first correction information for correcting the brightness information based on the inverted brightness information (S700), a step of applying the first correction information to the brightness information to generate converted brightness information (S800), and a step of generating an image corrected using the converted brightness information, saturation information, and hue information (S900).

[0048] Here, in the step of acquiring brightness information, saturation information, and hue information based on an original image (S500), the original image may be each of a left image, a center image, and a right image acquired from a plurality of cameras arranged in a row, or when used in conjunction with a technique for synthesizing into one image, it may be an image acquired or generated at that step, for example, a final wide image. The original image may be represented by RGB values, HSB (hue-saturation-brightness) values, or HLS (hue-lightness-saturation) values ​​for each pixel, and when represented by RGB values, it may be converted and stored into HLS values ​​indicating hue information, lightness information, and saturation information, respectively.

[0049] In the step S600 of inverting the acquired brightness information to acquire inverted brightness information of the present invention, brightness information is generally expressed in 256 levels, with the darkest being 0 and the brightest being 255 for each individual pixel, and for example, the inverted brightness information of brightness information 12 can be 255-12=243. This can be understood from the fact that when Fig. 5(c) is the brightness information acquired from the original image of Fig. 5(a), Fig. 5(d) is the inverted brightness information.

[0050] Next, the step of acquiring first correction information for correcting the brightness information based on the inverted brightness information (S700) includes acquiring first correction information for correcting the brightness information that is not 0 only when the inverted brightness information is equal to or greater than a threshold value. That is, when the inverted brightness information is equal to or greater than a threshold value, the first correction information for correcting the brightness information may be larger than when the inverted brightness information is equal to or less than the threshold value, and for example, when the inverted brightness information is equal to or less than the threshold value, the first correction information for the brightness information may be 0. Such brightness adjustment to an appropriate level may be useful when it is not necessary to brighten the portion 60a that is not a dark region such as 50a in FIG. 5(a) that satisfies a predetermined standard. Here, the threshold value may be experimentally defined as necessary.

[0051] Next, a step S800 of applying the first correction information thus obtained to the brightness information to generate converted brightness information and a step S900 of generating a corrected image using the converted brightness information, saturation information, and hue information are performed.

[0052] Here, the image processing method of the present invention may further include a step of obtaining second correction information for correcting chroma information based on the inverted luminance information in order to adjust the chroma saturation that has become low or high due to the luminance adjustment to an appropriate level, and a step of generating converted chroma information by applying the second correction information to the chroma information, and in this case, the chroma information used in the step of generating an image corrected with the converted luminance information, chroma information, and hue information may be replaced with the converted chroma information. As in the step of obtaining first correction information for correcting luminance information (S700), the method may include a step of obtaining second correction information for correcting chroma information that is not 0 only when the inverted luminance information is equal to or greater than a threshold value. That is, when the inverted luminance information is equal to or greater than a threshold value, the second correction information for correcting chroma information may be larger than when the inverted luminance information is equal to or less than the threshold value, and for example, when the inverted luminance information is equal to or less than the threshold value, the second correction information for chroma information may be 0. This may be useful when it is not necessary to clarify even a minute region such as 60b in FIG. 5(b) that satisfies a predetermined criterion. Here, the threshold value may be experimentally defined as necessary. Of course, the threshold value as a criterion for defining the first correction information and the threshold value as a criterion for defining the second correction information can be set differently, and the first correction information and the second correction information can also be defined differently.

[0053] Therefore, when the technology for minimizing color and shape distortion in an original image and processing only dark areas to be clearly seen based on the image processing method of the present invention is applied to, for example, a ship cleaning robot, it is possible to obtain an output image as shown in FIG. 5(b), which results in a significant reduction in fatigue of the robot operator.

[0054] Meanwhile, ships have parts that cannot be moved or that may collide when cleaning the ship's surface using a ship cleaning robot, such as the bilge keel, anode, bow thruster, and sea chest. The robot operator can know the robot's location through the robot's position estimation system and carefully maneuver before reaching the relevant parts.

[0055] In addition to slime and seaweed, shelled fouling organisms such as barnacles and mussels live on the surface of ships. When the shelled fouling organisms are small, there is no major problem, but when they live on a large scale, the robot may collide with them and stop, or the robot may run over shelled fouling organisms that have not been removed, causing the robot to detach from the surface of the ship. In the case of slime or seaweed, cleaning is possible at maximum running speed, but in the case of shelled fouling organisms, cleaning must be carried out at a slower speed or the number of revolutions of the cleaning brush must be increased. Therefore, the robot operator must appropriately control the robot's running speed and the number of revolutions of the cleaning brush according to the situation, but it is very difficult to concentrate on operating for long periods of time.

[0056] Therefore, in order to reduce the fatigue of robot operators who operate the robot for long periods of time, artificial intelligence technology is needed that analyzes the images from the camera installed on the robot and automatically adjusts the robot's movement speed and the rotation speed of the cleaning brush.

[0057] The artificial intelligence is a program that runs on a PC and is created through learning such as deep learning and machine learning, and can be included in the robot operating program or installed in a separately installed PC. Here, the robot operating program generally refers to a necessary program required for a robot operator to generally operate a robot, such as commanding a robot to operate, displaying an image from a camera installed in the robot on a monitor, and displaying data from a sensor installed in the robot on a monitor. In particular, the robot operating program for a ship cleaning robot capable of executing the image processing method of the present invention defines the robot's moving speed and the number of rotations of the cleaning brush according to the situation, and when the operating program is executed, the robot's moving speed and the number of rotations of the cleaning brush are adjusted according to the following control model.

[0058] Therefore, in order to reduce the fatigue of the robot operator who operates the robot for a long time, an artificial intelligence technology that automatically adjusts the robot's moving speed and the number of rotations of the cleaning brush is required. In this case, such an artificial intelligence technology can be used separately from the technology that makes it easy to identify dark areas in the camera image and / or the technology that combines the images into one image, or can be used in conjunction with the above technology.

[0059] Thus, the robot control method according to the present invention may include a step of constructing a control model that adjusts the moving speed of the robot and the rotation speed of the cleaning brush by learning any one of the images acquired or generated by the image processing method, and a step of adjusting the moving speed of the robot and the rotation speed of the cleaning brush based on a final wide image generated by processing images acquired from a plurality of cameras arranged on the robot by the image processing method using the constructed control model.

[0060] In addition, the step of adjusting the moving speed of the robot and the rotation speed of the cleaning brush may include a step of displaying to the operator of the robot that the moving speed of the robot and the rotation speed of the cleaning brush are being adjusted, and these displays may include using at least one of text, graphics, vibration, sound, and lighting.

[0061] Meanwhile, the image processing method according to an embodiment of the present invention may have an aspect of being entirely hardware, entirely software, or partially hardware and partially software. In this specification, terms such as "unit", "module", "device" or "system" are intended to refer to a combination of hardware and software driven by the corresponding hardware. For example, hardware may be a computing device capable of processing data including a central processing unit (CPU), a graphic processing unit (GPU) or other processor. In addition, hardware may refer to a running process, object, executable, thread of execution, program, etc.

[0062] Meanwhile, the methods according to the embodiments of the present invention may be embodied at least in part in the form of program instructions and recorded in a computer-readable recording medium, for example, with a program product comprising a computer-readable medium containing program code, which may be executed by a processor to perform any or all of the steps, operations, or processes described therein.

[0063] The computer readable recording medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions recorded on the medium may be specially designed and configured for the embodiments, or may be known and available to those skilled in the art of computer software. Examples of computer readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code, such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, and the like. The hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0064] In summary, the image processing method of the present invention for processing a plurality of images into one image can reduce the fatigue of a robot operator by combining a plurality of camera images into one image. Also, the robot control method using the image processing method of the present invention can reduce the fatigue of a robot operator and clean the ship more cleanly.

[0065] The terms used herein, particularly in the claims (e.g., the body of the claims), are generally intended as "open" terms (e.g., "including" should be interpreted as "including but not limited to," "having" as "having at least or more," and "including" as "including but not limited to"). When a specific number is intended for an introduced claim recitation, such intent will be expressly recited in the claim, and in the absence of such recitation, it will be understood that no such intent exists.

[0066] While only certain features of the invention have been illustrated and described herein, various modifications and changes can occur to those skilled in the art, and it is therefore to be understood that the appended claims are intended to cover all such changes and modifications which fall within the true spirit of the invention.

Claims

1. A video processing method for processing a plurality of videos into one video, comprising: acquiring at least a left image, a center image, and a right image from a plurality of cameras arranged in a row; generating a top-view left image, a top-view center image, and a top-view right image from the left image, the center image, and the right image, respectively; generating a wide top-view image by synthesizing the top-view left image, the top-view center image, and the top-view right image; and 2. An image processing method comprising: generating and outputting the wide top-view image as a final wide image, The step of acquiring at least a left image, a center image, and a right image from the plurality of cameras arranged in a row includes: For each of the left image, the center image, and the right image, obtaining brightness information, saturation information, and hue information based on any one of the images obtained or generated in the previous step; inverting the acquired brightness information to acquire inverted brightness information; obtaining first correction information for correcting the luminance information based on the inverted luminance information; obtaining second correction information for correcting chroma information based on the inverted lightness information; applying the first correction information to the luminance information to generate converted luminance information; applying the second correction information to the chroma information to generate transformed chroma information; and generating an image corrected using the converted lightness information, the converted saturation information, and the converted hue information; The step of obtaining first correction information for correcting the luminance information or obtaining second correction information for correcting the chroma information based on the inverted luminance information includes: For each pixel constituting the final wide image, Only when the inverted luminance information is equal to or greater than a threshold, a first correction information for the luminance information or a second correction information for the chroma information is obtained that is not 0.

23. A video processing method comprising:

2. 2. The image processing method of claim 1, wherein each of the left image, the center image, the right image, and the final wide image is a perspective view image having one vanishing point, and each of the top-view left image, the top-view center image, the top-view right image, and the wide top-view image is a top-view image viewed from above downwards without a vanishing point.

3. the generating step of generating a top-view left image, a top-view center image, and a top-view right image from the left image, the center image, and the right image, respectively, and the generating step of generating and outputting the wide top-view image as a final wide image include Using perspective transformation, 3. The image processing method of claim 2, wherein, when the perspective transformation uses a transformation matrix, a transformation matrix used in the step of generating the left image, the center image, and the right image as a top-view left image, a top-view center image, and a top-view right image, respectively, is an inverse matrix of a transformation matrix used in the step of generating and outputting the wide top-view image as a final wide image.

4. generating a wide top-view image by synthesizing the top-view left image, the top-view center image, and the top-view right image, 4. The image processing method of claim 3, further comprising the step of: processing a boundary area in an area where at least two of the top-view left image, the top-view center image, and the top-view right image overlap with each other semi-transparently, and removing the remaining area.

5. The image processing method according to claim 1 , wherein the first correction information and the second correction information are different from each other.

6. A computer-readable recording medium having a program for carrying out the method according to any one of claims 1 to 5 recorded thereon.

7. A computer program stored on a medium for executing the method according to any one of claims 1 to 5 in combination with hardware.

8. A step of constructing a control model for adjusting the moving speed of the robot and the rotation speed of the cleaning brush by learning any one of the images acquired or generated by the image processing method of any one of claims 1 to 5; A robot control method comprising the step of: utilizing the constructed control model, and adjusting the moving speed of the robot and the number of rotations of a cleaning brush based on a final wide image generated by processing images acquired from a plurality of cameras arranged on the robot by the image processing method of any one of claims 1 to 5.

9. The step of adjusting the moving speed of the robot and the rotation speed of the cleaning brush comprises: providing an indication to an operator of the robot that the moving speed of the robot and the number of rotations of the cleaning brush are being adjusted; 9. The method of claim 8, wherein such display includes utilizing at least one of text, graphics, vibration, sound, and lighting.

Images