Farmland area detection system for autonomous driving

Abstract

This farmland area detection system for autonomous driving may: classify image data including farmland, ridges, and roads and obtained by capturing a farmland area into classes for each pixel by using semantic segmentation; convert the image data into a top view image by executing a Bird Eye View algorithm that maps, to a mask image, GPS coordinates of each pixel calculated from the image data; and set an autonomous driving path by calculating, from the top view image, corner vertex coordinates of a corner area on a driving path on which an unmanned autonomous driving device is driven. According to the present invention, the unmanned autonomous driving device can map, by using the captured image, a farmland area by calculating corner vertex coordinates for each corner while being driven, thereby generating an efficient and accurate autonomous driving path.

Description

Cultivation area detection system for autonomous driving

[0001] The present invention relates to a system for detecting cultivated land areas for autonomous driving, and more specifically, to a system for detecting cultivated land areas for autonomous driving that classifies class for each pixel using semantic segmentation of image data including cultivated land, ridges, and roads captured in a cultivated land area, converts the image into a top-view image by performing a Bird Eye View algorithm that maps the GPS coordinates of each pixel calculated from the image data to a mask image, and calculates the corner vertex coordinates of the corner area on the driving path of an unmanned autonomous driving device from the top-view image to set an autonomous driving path.

[0002] The conventional method for mapping cultivated land areas is a technology for generating autonomous driving paths, in which a driver equipped with a GPS device rides a tractor, drives the tractor to a corner of the cultivated land, and then directly presses a button on the GPS device to input location coordinates. By receiving the location coordinates of N corners of the cultivated land in this way, the area of ​​the cultivated land is calculated, and an autonomous driving path for the tractor can be generated based on these location coordinates.

[0003] However, conventional methods for mapping cultivated land areas involve manually inputting locations while moving through the cultivated land. This method has the problem that errors occur in calculating the cultivated land area when the cultivated land area is very large, as the location where each person presses the GPS device button varies. Additionally, it has the disadvantage of being inefficient because people have to move long distances in person.

[0004] The present invention aims to provide a system for detecting a cultivated land area for autonomous driving that classifies classes for each pixel using semantic segmentation of image data including cultivated land, ridges, and roads captured in a cultivated land area, converts the image into a top-view image by performing a Bird Eye View algorithm that maps the GPS coordinates of each pixel calculated from the image data to a mask image, and calculates the corner vertex coordinates of the corner area on the driving path of an unmanned autonomous driving device from the top-view image to set an autonomous driving path.

[0005] A cultivated land area detection system for autonomous driving according to the features of the present invention for achieving the above objective comprises: a camera image acquisition unit that captures a cultivated land area using a camera module mounted on an unmanned autonomous driving device and generates image data including cultivated land, ridges, and roads; a semantic segmentation processing unit that provides the generated image data as input to a trained deep learning model to distinguish a class (cultivated land, ridges, roads) for each pixel and generates a mask image; a GPS (Global Positioning System) module that calculates the center coordinates of the unmanned autonomous driving device; an IMU (Inertial Measurement Unit) sensor module that calculates the tilt and direction of the unmanned autonomous driving device and calculates the tilt and direction of the camera module; and a BEV (Bird Eye View) conversion unit that performs a Bird Eye View algorithm that calculates the GPS coordinates of each pixel in the image data based on the center coordinates of the unmanned autonomous driving device and maps the GPS coordinates for each pixel to the mask image. And it may include a corner vertex coordinate calculation unit that calculates the corner vertex coordinates of the cultivated land, ridge, and road at each corner area where the driving path of the unmanned autonomous driving device turns, using the pixel-by-pixel GPS coordinates while the unmanned autonomous driving device is driving.

[0006] The BEV conversion unit converts image data including the cultivated land, ridges, and roads into a top-view image by performing a Bird Eye View algorithm, and can generate a GPS map that distinguishes the cultivated land, ridges, and roads using the GPS location and direction of the top-view image. It may further include a control unit that sets an autonomous driving path using the generated GPS map.

[0007] The corner vertex coordinate calculation unit calculates the corner vertex coordinates of a cultivated field on the driving path of the unmanned autonomous driving device using the pixel-by-pixel GPS coordinates in the top-view image, and calculates the corner vertex coordinates of the ridge and the corner vertex coordinates of the road based on the calculated corner vertex coordinates of the cultivated field.

[0008] With the above-described configuration, the present invention can map a cultivated land area by calculating the corner vertex coordinates at each corner while an unmanned autonomous driving device drives using captured images, thereby enabling the generation of an efficient and accurate autonomous driving path.

[0009] The present invention has the effect of efficiently mapping a cultivated land area by moving a short distance, as it can automatically obtain corner vertex coordinates while driving at a position a certain distance inward from the boundary of the cultivated land using an unmanned autonomous driving device.

[0010] FIG. 1 is a diagram showing the configuration of a cultivated land area detection system for autonomous driving according to an embodiment of the present invention.

[0011] FIG. 2 is a diagram showing the configuration of a cultivated land area detection device according to an embodiment of the present invention.

[0012] FIG. 3 is a diagram showing the configuration of a semantic segmentation processing unit according to an embodiment of the present invention.

[0013] FIG. 4 is a diagram showing an example of a mask image, which is a semantic segmentation result according to an embodiment of the present invention.

[0014] FIG. 5 is a diagram showing a method for detecting a cultivated land area for autonomous driving according to an embodiment of the present invention.

[0015] FIGS. 6 to 11 are drawings illustrating the process of generating corner vertex coordinates on a driving path of a tractor according to an embodiment of the present invention.

[0016] FIG. 12 is a drawing showing an example of a top-view image of a Bird Eye View according to an embodiment of the present invention.

[0017] The present invention is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0018] Terms such as first, second, A, B, etc., may be used to describe various components, but said components shall not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0019] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0020] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0021] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0022] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate an overall understanding of the present invention, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0023] Hereinafter, the cultivation area detection system for autonomous driving according to the present invention will be described with reference to the attached drawings.

[0024] FIG. 1 is a diagram showing the configuration of a farmland area detection system for autonomous driving according to an embodiment of the present invention, FIG. 2 is a diagram showing the configuration of a farmland area detection device according to an embodiment of the present invention, FIG. 3 is a diagram showing the configuration of a semantic segmentation processing unit according to an embodiment of the present invention, FIG. 4 is a diagram showing an example of a mask image which is a semantic segmentation result according to an embodiment of the present invention, and FIG. 5 is a diagram showing a farmland area detection method for autonomous driving according to an embodiment of the present invention.

[0025] FIGS. 6 to 11 are drawings illustrating the process of generating corner vertex coordinates on a driving path of a tractor according to an embodiment of the present invention, and FIG. 12 is a drawing illustrating an example of a top-view image of a Bird Eye View according to an embodiment of the present invention.

[0026] A cultivation area detection system (100) for autonomous driving according to an embodiment of the present invention may be equipped with a Global Positioning System (GPS) module (120), an Inertial Measurement Unit (IMU) sensor module (122), and a cultivation area detection device (130) mounted on an agricultural machine (110). Although the agricultural machine of the present invention is exemplified by a tractor, it is not limited thereto, and any mobile device capable of unmanned autonomous driving may be possible.

[0027] A cultivated land area detection device (130) according to an embodiment of the present invention may include a camera image acquisition unit (131), a semantic segmentation processing unit (132), a BEV (Bird Eye View) conversion unit (135), a corner vertex coordinate calculation unit (136), a control unit (137), a data storage unit (138), and a display unit (139).

[0028] The camera image acquisition unit (131) can install a camera module on agricultural machinery such as a tractor and generate image data including a cultivated field (10), a ridge (20), and a road (30) by means of the camera module (S100). The camera module can be used with the same concept as a camera that captures images.

[0029] The semantic segmentation processing unit (132) may include a data generation unit (133) and a deep learning model unit (134).

[0030] The semantic segmentation processing unit (132) can generate a mask image by providing the generated image data as input to a trained deep learning model and distinguishing the class (farmland, ridge, road) for each pixel (S110).

[0031] The data generation unit (133) can label the image data collected from the camera image acquisition unit (131) by class (cultivated land (10), ridge (20), road (30)) using a labeling tool such as LabelMe, CVAT, or RectLabel for each pixel, and can generate a dataset by pairing the labeled image data with the original image data and separating them into a training dataset, a validation dataset, and a test dataset. For example, the cultivated land (10) can be displayed in sky blue, the ridge (20) in orange, and the road (30) in light green.

[0032] The deep learning model unit (134) selects a deep learning model for semantic segmentation (artificial intelligence algorithm) and inputs labeled image data into the selected deep learning model to train it. Here, the deep learning model may include U-Net designed for pixel-unit segmentation in images, DeepLab, a segmentation model that enhances edge detection and multi-scale processing, SegNet, MobileNet, etc., which restores details of the original image data after segmentation.

[0033] The deep learning model unit (134) learns using a loss function that minimizes the difference between the predicted pixel class and the actual class using cross-entropy loss.

[0034] The deep learning model unit (134) calculates a loss function between the actual label and the output using forward propagation to generate the output (pixel class prediction) of the deep learning model through input data.

[0035] The deep learning model unit (134) updates weights based on the calculated loss function and learns model parameters.

[0036] The deep learning model unit (134) performs repeated learning on the entire dataset multiple times to minimize the loss function.

[0037] The deep learning model unit (134) can evaluate the performance of the learned deep learning model using a verification dataset. In other words, the deep learning model unit (134) can calculate performance indicators using Intersection over Union (IoU), which evaluates how well the predicted area of ​​each class matches the actual area, Mean Accuracy, which represents the ratio of correctly predicted pixels among all pixels, and Pixel Accuracy, which represents the average accuracy for each class.

[0038] The deep learning model unit (134) can determine whether overfitting occurs by checking the difference in performance between the training data and the validation data, and if overfitting occurs, it can apply Dropout, data augmentation, etc.

[0039] As illustrated in FIG. 4, the deep learning model unit (134) provides image data captured by the camera image acquisition unit (131) as input to the learned deep learning model, and the deep learning model can generate a mask image by predicting a class (farmland (10), ridge (20), road (30)) for each pixel of the captured image data. The mask is distinguished by the colors of farmland (sky blue), ridge (orange), and road (light green).

[0040] The GPS module (120) is mounted on the tractor (110) and communicates in real time with the base station to calculate the absolute position coordinates of the tractor (110).

[0041] The IMU sensor module (122) is mounted on the tractor (110) and can generate IMU data, which is the distance traveled, speed, angle, and direction change of the tractor (110), by measuring inertial information including an accelerometer that measures the linear acceleration of the tractor (110) to calculate the direction and speed of movement and a gyroscope that measures the rotational angular velocity of the tractor (110) to calculate the change in direction (angle).

[0042] The IMU sensor module (122) can generate IMU data, which is the speed, angle, and direction change of the camera module of the camera image acquisition unit (131).

[0043] The GPS module (120) can calculate the center coordinates of the tractor (110), the IMU sensor module (122) can calculate the tilt and direction of the tractor (110), and the tilt and direction of the camera module (S120).

[0044] The BEV conversion unit (135) can calculate the GPS coordinates of each pixel in the image data based on the center coordinates of the tractor (110) and perform a Bird Eye View algorithm that maps the GPS coordinates of each pixel to a mask image (S130).

[0045] The BEV conversion unit (135) receives the center coordinates of the tractor (110) from the GPS module (120) and stores them in the data storage unit (138), and can receive the tilt and direction of the tractor (110) and the tilt and direction of the camera module from the IMU sensor module (122) and store them in the data storage unit (138). The tilt of the tractor (110) equipped with the camera module refers to the angle of forward and backward tilt of the tractor (110) (Pitch) and the angle of left and right tilt of the tractor (110) (Roll), and the direction may refer to the rotation direction (Yaw) of the tractor (110). The tilt of the camera module includes the angle of forward and backward tilt of the camera module (Pitch) and the angle of left and right tilt of the camera module (Roll), and the direction may refer to the rotation direction (Yaw) of the camera module.

[0046] The BEV conversion unit (135) can determine the area occupied on the actual ground and image data captured by the camera module by calculating the camera field of view (FOV) using the following mathematical formula 1. The image data may be an image including a cultivated field (10), a ridge (20), and a road (30).

[0047] [Mathematical Formula 1]

[0048]

[0049] Here, Horizontal FOV is the horizontal field of view of the camera module, Vertical FOV is the vertical field of view of the camera module, Sensor Width is the horizontal length of the camera module, Sensor Height is the vertical length of the camera module, and Focal Length is the focal length of the camera lens (distance from the center of the lens to the camera module).

[0050] FOV is used to calculate the area (ground size) that is actually observable based on the camera's field of view and shooting height.

[0051] The BEV conversion unit (135) determines how far each pixel (u, v) of the image data is from the coordinates of the center pixel of the image data (e.g., 960, 540, etc.) relative coordinates of each pixel ( ) can be calculated. The center pixel of the image coincides with the center of the camera module's field of view.

[0052] [Mathematical Formula 2]

[0053]

[0054] Here, u and v are image pixel coordinates (u: horizontal pixels, v: vertical pixels), Image Width is the horizontal resolution (number of pixels) of the image, and Image Height is the vertical resolution (number of pixels), and is a relative coordinate indicating how far it is from the coordinates of the center pixel of the image data.

[0055] The BEV conversion unit (135) uses the following mathematical formula 3 to determine the angle of each pixel ( A direction vector can be generated by calculating ).

[0056] The BEV conversion unit (135) can calculate the field of view covered by one pixel using the resolution and FOV of the image data. More specifically, the BEV conversion unit (135) can calculate the Horizontal Angle per Pixel by dividing the Horizontal FOV by the Image Width, and calculate the Vertical Angle per Pixel by dividing the Vertical FOV by the Image Height.

[0057] [Mathematical Formula 3]

[0058]

[0059] Here, is the horizontal angle that a specific pixel occupies from the center of the camera module ( ) and vertical angle( ) and Horizontal Angle per Pixel is the angle occupied in the horizontal direction of each pixel, and Vertical Angle per Pixel is the angle occupied in the vertical direction of each pixel. indicates how many degrees the corresponding pixel is horizontally offset from the center of the image, and indicates how many degrees the corresponding pixel is vertically offset from the center of the image, and represents the relative coordinates from the center of the image.

[0060] represents the angular position of the pixel from the center of the actual camera module, and this angle plays an important role when converting the pixel to ground coordinates or creating a Bird Eye View.

[0061] The BEV conversion unit (135) is the angle of each pixel ( The pixel direction vector D can be calculated by substituting ) into the following mathematical formula 4.

[0062] The pixel direction vector can be converted into the (X, Y, Z) form in the camera coordinate system.

[0063] [Mathematical Formula 4]

[0064]

[0065] Here, D represents a vector indicating the pixel direction relative to the center of the camera module. This pixel direction vector D can be used to define the direction of each pixel in 3D space relative to the camera module's coordinate system. D may be a unit direction vector indicating the direction each pixel points in the camera coordinate system. This can be used when projecting from the image plane (2D) of the camera module into 3D space.

[0066] D in 3D space indicates the direction in which each pixel faces from the center of the camera module. Through this, the direction in 3D space that each pixel points to in the 2D image seen by the camera can be calculated.

[0067] Here, tan( ) is the tangent value for the horizontal angle of the pixel, and tan( ) is the tangent value for the vertical angle of the pixel, and 1 represents the unit depth (distance) between the camera module and the image plane.

[0068] tan( ) and tan( ) calculates the gradient of the direction the pixel points in 3D space. The 3D direction of each pixel is expressed by Equation 4, which can be used to convert the pixel into a 3D coordinate system. The direction vector D indicates the direction in which each pixel faces from the center of the camera module.

[0069] The pixel direction vector D is an important element representing the direction of a pixel in the camera coordinate system and is used to project 2D image data into a 3D world or to calculate coordinates on the ground in the Bird Eye View algorithm.

[0070] In other words, the pixel direction vector D can calculate the direction of a specific pixel from the camera center and can be used to calculate the position of the pixel on the ground.

[0071] The Bird Eye View transformation can calculate the position a pixel points to on the ground by combining the direction vector D and the position (height) of the camera module. The pixel direction can be converted to the world coordinate system by applying the direction vector D to rotation and transformation.

[0072] The BEV conversion unit (135) can convert the direction vector into a world coordinate system by applying the camera's tilt (Pitch, Roll) and rotation (Yaw) using IMU data. The most representative example of a world coordinate system may be a GPS coordinate system.

[0073] [Mathematical Formula 5]

[0074]

[0075] Here, is the world coordinate after the pixel coordinates have been corrected, and is a rotation matrix generated based on IMU data, representing the tilt and direction, respectively, and represents the value obtained by converting the pixel coordinates calculated in Equation 4 into a camera coordinate system vector.

[0076] The BEV conversion unit (135) receives each rotation matrix based on the tilt and direction of the tractor (110) from the IMU sensor module (122). ) can be calculated.

[0077] The BEV conversion unit (135) can calculate the intersection point with the ground using the height (h) between the camera module and the ground using the following mathematical formula 6, and convert pixel coordinates into GPS coordinates. The BEV conversion unit (135) can calculate the GPS coordinates corresponding to each pixel ( Saves ).

[0078] [Mathematical Formula 6]

[0079]

[0080] Here, is the actual GPS coordinate of the converted pixel, and is the center GPS coordinates (latitude, longitude) of the tractor (110), and h is the height between the camera module and the ground, represents the pixel direction vector after correcting the camera coordinate system calculated by mathematical formula 5.

[0081] The BEV conversion unit (135) maps the obtained pixel-by-pixel GPS coordinates to the semantic segmentation result, which is a mask image generated by the deep learning model unit (134). In other words, the BEV conversion unit (135) maps the corresponding GPS coordinates to the semantic segmentation classification result of each pixel (e.g., cultivated land (10), ridge (20), road (30)) to visualize the position each pixel occupies in real space as a Bird Eye View.

[0082] As illustrated in FIG. 12, the BEV conversion unit (135) can perform the task of mapping the GPS coordinates of each pixel in image data including a cultivated field (10), a ridge (20), and a road (30) to a class (cultivated field (10), a ridge (20), and a road (30)). This task is called Bird Eye View.

[0083] The Bird Eye View algorithm can indicate which area each pixel corresponds to in real space based on the center coordinates of the tractor (110).

[0084] As illustrated in FIG. 12, the BEV conversion unit (135) can generate a Bird Eye View generation result, that is, a top-view image viewed from above. The top-view image generates a 2D or 3D map image in a top-down view, and classes such as cultivated land (sky blue), ridges (orange), and roads (light green) can be distinguished and displayed by color. These color codes reflect the semantic segmentation results exactly.

[0085] The BEV conversion unit (135) can generate a GPS map that distinguishes between a cultivated field (10), a ridge (20), and a road (30) using the top view image's GPS location and direction.

[0086] The generated top-view image is represented in the form of a map showing the working environment of the cultivated land (10), ridges (20), and roads (30) around the tractor from above, allowing the boundaries to be clearly identified or the worker to easily grasp the layout of the current environment and the relative position of each pixel. Each pixel can represent a specific location in the actual space.

[0087] The top view image of the Bird Eye View is generated based on the camera module's height, focal length, and shooting angle, so it can reflect the actual size and aspect ratio. The top view image visualizes pixels in color and displays GPS coordinates as a grid, allowing the user to accurately identify the location of each area. The top view image can be utilized for various spatial analyses, including distance calculations, and when generated based on GPS coordinates and image data, it can calculate the relationship between the size of each pixel and the distance to the actual ground.

[0088] The top-view image may include cultivated land (10), ridges (20), roads (30), respective boundary lines separating them, and GPS coordinates of each pixel.

[0089]

[0090] A method for generating corner vertex coordinates on a driving path on which a tractor (110) travels according to an embodiment of the present invention is as follows.

[0091] The corner vertex coordinate calculation unit (136) can calculate the corner vertex coordinates of the cultivated land (10), ridge (20), and road (30) at each corner area where the driving path of the tractor (110) bends, using pixel-by-pixel GPS coordinates while the tractor (110) is driving.

[0092] The corner vertex coordinate calculation unit (136) can calculate the corner vertex coordinates while driving the Nth driving path (S140).

[0093] The corner vertex coordinate calculation unit (136) converts the GPS coordinates of each pixel in the image data, including the cultivated land (10), ridge (20), and road (30) generated by the BEV conversion unit (135), into a planar coordinate system. The method of converting GPS coordinates into planar coordinates is a known technique, so a detailed explanation is omitted.

[0094] The corner vertex coordinate calculation unit (136) calculates the corner vertex coordinates of the cultivated land (10) on the driving path of the tractor (110) using pixel-by-pixel GPS coordinates in the top-view image, and then calculates the corner vertex coordinates of the ridge (20) and the corner vertex coordinates of the road (30) based on this. This process can be performed by calculating the spatial relationship with other boundary areas (ridge (20) and road (30)) based on the data of the tractor's (110) driving path.

[0095] As illustrated in FIG. 10, the corner vertex coordinate calculation unit (136) can calculate a first angle between the pixel coordinates of the boundary line of the cultivated land (10) and the pixel coordinates of the tractor (110) on the road using the following mathematical formula 7.

[0096] [Mathematical Formula 7]

[0097]

[0098] Here, is the x and y coordinates of the first point on the straight line of the tractor's driving path, and may be the current position or reference point of the tractor (110), and represents the x and y coordinates of the second point on the straight line of the boundary of the cultivated land (10).

[0099] The corner vertex coordinate calculation unit (136) can detect a corner of the cultivated land that turns at the boundary line of the cultivated land when the calculated first angle changes by more than a predetermined first reference threshold (e.g., 30 degrees or more).

[0100] The corner vertex coordinate calculation unit (136) can store the first GPS coordinates corresponding to the points of the corners of the detected cultivated land and the planar coordinates of the first GPS coordinates in the data storage unit (138).

[0101] As illustrated in FIG. 10, the corner vertex coordinate calculation unit (136) can calculate a second angle between the pixel coordinates of the boundary line of the ridge (20) and the pixel coordinates of the tractor (110) on the road using the aforementioned mathematical formula 7. Here, is the x and y coordinates of the first point on the straight line of the tractor's driving path, and may be the current position or reference point of the tractor (110), and represents the x and y coordinates of the second point on the straight line of the boundary line of the ridge (20).

[0102] The corner vertex coordinate calculation unit (136) can detect a corner bending at the corner boundary line when the calculated second angle changes to a predetermined second threshold value (e.g., 30 degrees or more).

[0103] The corner vertex coordinate calculation unit (136) can store the second GPS coordinates corresponding to the points of the detected ridge corners and the planar coordinates of the second GPS coordinates in the data storage unit (138).

[0104] As illustrated in FIG. 10, the corner vertex coordinate calculation unit (136) can calculate a third angle between the pixel coordinates of the boundary line of the road (30) and the pixel coordinates of the tractor (110) on the road using the aforementioned mathematical formula 7. Here, is the x and y coordinates of the first point on the straight line of the tractor's driving path, and may be the current position or reference point of the tractor (110), and represents the x and y coordinates of the second point on the straight line of the boundary line of the road (30).

[0105] The corner vertex coordinate calculation unit (136) can detect a road corner that turns at the road boundary line when the calculated third angle changes by more than a predetermined third threshold value (e.g., 30 degrees or more).

[0106] The corner vertex coordinate calculation unit (136) can store the third GPS coordinates corresponding to the points of the detected road corners and the planar coordinates of the third GPS coordinates in the data storage unit (138).

[0107] The control unit (137) calculates the distance between the vertex coordinates of the cultivated land (10) and the vertex coordinates of the ridge (20) using a planar coordinate system (x,y) and the Euclidean distance formula, calculates the distance between the vertex coordinates of the cultivated land (10) and the vertex coordinates of the road (30), and calculates the distance between the vertex coordinates of the ridge (20) and the vertex coordinates of the road (30). The distance between the vertex coordinates is because boundaries must be set to accurately define the working area of ​​the cultivated land (10).

[0108] The control unit (137) controls the semantic segmentation result (Fig. 4), which is a mask image, and the top view image of the Bird Eye View (Fig. 12) to be output to the display unit (139).

[0109] The control unit (137) can generate multiple waypoints, which are specific point locations, at regular intervals along the tractor's driving path. The waypoints are represented by GPS coordinates and are used as reference points to control movement direction, speed, rotation, etc. in an autonomous driving system. They can also be used to define points for performing tasks (e.g., sowing, harvesting, etc.) in addition to simple movement.

[0110] The control unit (137) can set the number or spacing of waypoints differently based on the width of the tractor (110), the turning radius of the tractor (110), and the width of the implement attached to the tractor (110).

[0111] The control unit (137) can set an autonomous driving path using the generated GPS map.

[0112] The control unit (137) calculates the corner vertex coordinates while the tractor (110) travels along the Nth driving path, and then compares N and M (e.g., 4). If it determines that N is less than or equal to M, it performs step S160 (N+1) and proceeds to step S120. If it determines that N is greater than M, it terminates.

[0113] As the tractor (110) travels along the first driving path, the second driving path, the third driving path, and the fourth driving path, it can calculate the corner vertex coordinates of the cultivated land (10), the corner vertex coordinates of the ridge (20), and the corner vertex coordinates of the road (30) at each corner.

[0114] The cultivated land area detection device (130) is characterized by generating an autonomous driving path for the tractor (110) by calculating corner vertex coordinates even without a person directly driving the tractor (110).

[0115] The tractor (110) can calculate corner vertex coordinates while driving at a position N meters away from the boundary of the cultivated land toward the center of the cultivated land. That is, if a person drives the tractor (110) and manually inputs the location coordinates, they must enter GPS coordinates by approaching the corner. In contrast, the present invention uses an unmanned autonomous driving device to automatically obtain corner vertex coordinates while driving at a certain distance inward from the boundary of the cultivated land, thereby allowing for efficient mapping of the cultivated land area by moving a short distance. Therefore, the driving path can be formed at a position N meters away from the boundary of the cultivated land toward the center of the cultivated land.

[0116] The technical features disclosed in each embodiment of the present invention are not limited to that embodiment only, and as long as they are not mutually incompatible, the technical features disclosed in each embodiment may be combined and applied to different embodiments.

[0117] Therefore, in each embodiment, the technical features are described primarily, but as long as the technical features are not mutually incompatible, they may be combined and applied together.

[0118] The present invention is not limited to the embodiments described above and the attached drawings, and various modifications and variations may be possible from the perspective of those skilled in the art to which the present invention belongs. Accordingly, the scope of the present invention should be defined not only by the claims of this specification but also by equivalents thereof.

Claims

1. A camera image acquisition unit that captures a cultivated land area using a camera module mounted on an unmanned autonomous driving device and generates image data including the cultivated land, ridges, and roads; A semantic segmentation processing unit that provides the above-mentioned generated image data as input to a trained deep learning model to distinguish a class (cultivated land, ridge, road) for each pixel and generate a mask image; A GPS (Global Positioning System) module for calculating the center coordinates of the above-mentioned unmanned autonomous driving device; An IMU (Inertial Measurement Unit) sensor module that calculates the tilt and direction of the above-mentioned unmanned autonomous driving device and calculates the tilt and direction of the above-mentioned camera module; A BEV (Bird Eye View) conversion unit that calculates the GPS coordinates of each pixel in the image data based on the center coordinates of the above-mentioned unmanned autonomous driving device and performs a Bird Eye View algorithm that maps the pixel-specific GPS coordinates to the mask image; and A corner vertex coordinate calculation unit that calculates the corner vertex coordinates of the cultivated land, ridge, and road at each corner area where the driving path of the unmanned autonomous driving device bends, using the pixel-by-pixel GPS coordinates while the unmanned autonomous driving device is driving. A farmland area detection system for autonomous driving including 2. In Paragraph 1, The above BEV conversion unit converts image data including the cultivated land, ridges, and roads into a top-view image by performing the Bird Eye View algorithm, and generates a GPS map that distinguishes the cultivated land, ridges, and roads using the GPS location and direction of the top-view image. A control unit that sets an autonomous driving path using the GPS map generated above. A farmland area detection system for autonomous driving that further includes 3. In Paragraph 2, The above edge vertex coordinate calculation unit is, Calculate the corner vertex coordinates of the cultivated land on the driving path of the unmanned autonomous driving device using the pixel-by-pixel GPS coordinates in the top-view image above, and calculate the corner vertex coordinates of the ridge and the corner vertex coordinates of the road based on the calculated corner vertex coordinates of the cultivated land. Cultivation area detection system for autonomous driving.

4. In Paragraph 2, The above edge vertex coordinate calculation unit is, The GPS coordinates of each pixel in the image data including the cultivated land, ridge, and road generated in the above BEV conversion unit are converted into a planar coordinate system, and a first angle between the pixel coordinates of the boundary line of the cultivated land and the pixel coordinates of the driving of the above unmanned autonomous driving device is calculated using the following mathematical formula 1, and if the calculated first angle changes by more than a predetermined first reference threshold, it is detected as a corner of the cultivated land that bends at the boundary line of the cultivated land, and the first GPS coordinates corresponding to the point of the detected corner of the cultivated land and the planar coordinates of the first GPS coordinates are stored. Cultivation area detection system for autonomous driving. [Mathematical Formula 1] Here, is the x and y coordinates of the first point on the straight line of the driving path of the unmanned autonomous driving device, and may be the current location or reference point of the unmanned autonomous driving device, is the x and y coordinates of the second point on the straight line of the cultivated land boundary.

5. In Paragraph 4, The above edge vertex coordinate calculation unit is, Calculate a second angle between the pixel coordinates of the boundary line of the ridge and the pixel coordinates on the driving side of the unmanned autonomous driving device using the above mathematical formula 1, and if the calculated second angle changes by more than a predetermined second reference threshold, detect it as a ridge corner that bends at the boundary line of the ridge, and store the second GPS coordinates corresponding to the point of the detected ridge corner and the planar coordinates of the second GPS coordinates. Cultivation area detection system for autonomous driving.

6. In Paragraph 5, The above edge vertex coordinate calculation unit is, Calculate a third angle between the pixel coordinates of the road boundary line and the pixel coordinates of the driving of the unmanned autonomous driving device using the above mathematical formula 1, and if the calculated third angle changes by more than a predetermined third reference threshold, detect it as a road corner bending at the road boundary line, and store the third GPS coordinates corresponding to the point of the detected road corner and the planar coordinates of the third GPS coordinates. Cultivation area detection system for autonomous driving.

7. In Paragraph 6, The above control unit is, Calculate the distance between the vertex coordinates of the above-mentioned cultivated land and the vertex coordinates of the above-mentioned ridge using the Euclidean distance formula, calculate the distance between the vertex coordinates of the above-mentioned cultivated land and the vertex coordinates of the above-mentioned road, and calculate the distance between the vertex coordinates of the above-mentioned ridge and the vertex coordinates of the above-mentioned road. Cultivation area detection system for autonomous driving.

8. In Paragraph 2, The above control unit is, Controls the output of the semantic segmentation result, which is the mask image, and the top-view image to the display unit, generates multiple waypoints, which are specific point locations, at regular intervals on the driving path of the unmanned autonomous driving device, and sets the number or interval of the waypoints differently based on the width of the unmanned autonomous driving device, the turning radius of the unmanned autonomous driving device, and the width of the implement coupled to the unmanned autonomous driving device. Cultivation area detection system for autonomous driving.

9. In Paragraph 3, The above driving path is formed at a location N meters away from the boundary of the cultivated land toward the center of the cultivated land. Cultivation area detection system for autonomous driving.

Images