Vision-based stable navigation signal generation method, device and equipment
By using a vision-based navigation signal generation method, a stable navigation signal for agricultural machinery is generated using a color camera and visual target recognition technology. This solves the problem of unstable scene information acquisition in the automatic driving of agricultural machinery and improves navigation accuracy and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-24
- Publication Date
- 2026-04-14
AI Technical Summary
The high cost and instability of scene information acquisition in agricultural machinery autonomous driving leads to a decrease in navigation accuracy, especially when there is a lack of standard feature objects in agricultural scenes.
A vision-based stable navigation signal generation method is adopted. Image information is acquired through a color camera, and visual target recognition technology is used to identify feature line masks, which are then converted into feature lines in the agricultural machinery coordinate system. This process is used for region division and updating, and navigation is performed by combining navigation lines to generate heading error and lateral error.
It reduced equipment costs, improved the efficiency and stability of information acquisition in agricultural machinery autonomous driving scenarios, reduced navigation jitter, and expanded the application scope of visual navigation methods.
Smart Images
Figure CN121855533A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of visual navigation technology, and in particular to a method, apparatus and device for generating stable navigation signals based on vision. Background Technology
[0002] In the agricultural sector, automation and intelligence are becoming future development trends. Currently, agricultural machinery automatic driving technology relies on additional, manually labeled scene information, which is not only costly, but also means that the acquired scene information is only valid for a period of time because agricultural scenes are constantly changing with crop growth. When the scene changes, the scene information needs to be acquired again, which further increases the cost of agricultural machinery automatic driving and reduces production efficiency.
[0003] Furthermore, the navigation control module is very sensitive to the input information. If the input information is not stable enough, significant jitter will occur, affecting the navigation accuracy of the agricultural machinery. In addition, compared with the road scenarios of autonomous driving in cars, the agricultural scenarios do not have standard features, or rather, the features are not standardized, which leads to the problem of data instability in the agricultural scenarios themselves.
[0004] Regarding the aforementioned technologies, the inventors discovered that existing agricultural machinery automatic driving systems suffer from problems such as high cost, time consumption, and instability in acquiring scene information. Summary of the Invention
[0005] To improve the efficiency and stability of information acquisition in agricultural machinery autonomous driving scenarios and reduce the cost of acquiring agricultural scenario information, this application provides a vision-based stable navigation signal generation method.
[0006] In one aspect, this application provides a vision-based method, apparatus, and device for generating stable navigation signals.
[0007] This application is achieved through the following technical solution:
[0008] A vision-based method for generating stable navigation signals includes the following steps:
[0009] Acquire image information from a color camera;
[0010] The image information is identified using visual target recognition technology to obtain a feature row mask;
[0011] The feature row mask is converted into feature lines in the agricultural machinery coordinate system. The number of feature lines within the wheel track range of the agricultural machinery is counted to determine the region width. The region width is then used to divide the region.
[0012] Update the feature lines in the agricultural machinery coordinate system according to the region division results and the inter-frame correspondence;
[0013] When a feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery, or when a feature line exists in the symmetrical area of the midpoint of the front axle of the agricultural machinery, the feature line is added to the feature line list.
[0014] Based on the list of feature lines, determine the navigation lines;
[0015] Based on the navigation lines and combined with the agricultural machinery direction lines, the heading error and lateral error are obtained and used for agricultural machinery navigation.
[0016] In a preferred embodiment, this application can be further configured to include the following steps:
[0017] When there is no feature line in the area closest to the midpoint of the front axle of the agricultural machinery, or when there is no matching feature line in the symmetrical area of the midpoint of the front axle of the agricultural machinery, the feature line in the agricultural machinery coordinate system is offset.
[0018] Add the feature lines in the offset agricultural machinery coordinate system that meet the conditions to the feature line list.
[0019] In a preferred embodiment, this application can be further configured such that the step of dividing the region using the region width includes:
[0020] When the number of feature lines within the wheel track range of the agricultural machinery is even, the symmetrical regions at the distance from the midpoint of the front axle of the agricultural machinery are matched with region IDs of ±1, ±2, ±3, ..., ±N, where N is a positive integer, based on the region interval of the region width.
[0021] When the number of feature lines within the wheel track range of the agricultural machinery is odd, the region ID of the region closest to the midpoint of the front axle of the agricultural machinery is initialized to 0, and the symmetrical regions with region ID 0 are successively ±1, ±2, ±3, ..., ±N, where N is a positive integer;
[0022] Add two symmetrical region IDs outside the wheel track range of agricultural machinery.
[0023] In a preferred embodiment, this application can be further configured such that the step of determining the navigation line based on the feature line list includes,
[0024]
[0025] In the formula, navi line For navigation lines, Len(V) represents the total number of feature lines in the feature line list, and V... line (i) is any feature line in the feature line list.
[0026] In a preferred embodiment, this application can be further configured such that the step of converting the feature row mask into feature lines in the agricultural machinery coordinate system includes:
[0027] Obtain the midpoint coordinates of each row of the feature row mask in the image coordinate system;
[0028] Based on the coordinates of each midpoint, and combined with straight line fitting technology, the feature row mask is converted into feature lines;
[0029] Based on the preset agricultural machinery parameters and camera parameters, the feature lines are transformed into the agricultural machinery coordinate system.
[0030] In a preferred embodiment, this application can be further configured to include the following steps:
[0031] If no feature lines are detected for M consecutive frames, it is determined that there are no feature lines in the region.
[0032] When a feature line is detected in M consecutive frames, the feature line of the previous frame is used as the feature line of the region.
[0033] Where M is a positive integer.
[0034] In a preferred embodiment, this application may be further configured such that, after the step of determining the navigation lines based on the feature line list, the method further includes,
[0035] The navigation lines are filtered based on the previous navigation lines.
[0036] In a preferred embodiment, this application may be further configured such that, after the step of obtaining the heading error and lateral error based on the navigation line and in combination with the agricultural machinery direction line for agricultural machinery navigation, it further includes,
[0037] Based on the previous heading error and the previous lateral error, the heading error and the lateral error are filtered.
[0038] Secondly, this application provides a vision-based stable navigation signal generation device.
[0039] This application is achieved through the following technical solution:
[0040] A vision-based stable navigation signal generation device, comprising,
[0041] An image module is used to acquire image information from a color camera;
[0042] The masking module is used to identify the image information using visual target recognition technology to obtain a feature row mask;
[0043] The partitioning module is used to convert the feature row mask into feature lines in the agricultural machinery coordinate system, count the number of feature lines within the wheel track range of the agricultural machinery, determine the region width, and use the region width to partition the region.
[0044] The update module is used to update the feature lines in the agricultural machinery coordinate system according to the region division results and the inter-frame correspondence.
[0045] The collection module is used to add the feature line to the feature line list when the feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery or when the feature line exists in the symmetrical area of the midpoint of the front axle of the agricultural machinery.
[0046] The navigation line module is used to determine navigation lines based on the feature line list;
[0047] The navigation signal module is used to obtain heading error and lateral error based on the navigation line and the agricultural machinery direction line for agricultural machinery navigation.
[0048] Thirdly, this application provides a computer device.
[0049] This application is achieved through the following technical solution:
[0050] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described vision-based stable navigation signal generation methods.
[0051] Fourthly, this application provides a computer-readable storage medium.
[0052] This application is achieved through the following technical solution:
[0053] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described vision-based stable navigation signal generation methods.
[0054] Fifthly, this application provides a computer program product.
[0055] This application is achieved through the following technical solution:
[0056] A computer program product includes a computer program that, when executed by a processor, implements the steps of any of the above-described vision-based stable navigation signal generation methods.
[0057] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following:
[0058] It acquires image information solely from a color camera, requiring no additional equipment such as LiDAR, depth cameras, inertial navigation systems, or positioning devices, resulting in low equipment costs. Visual target recognition technology is used to identify image information, obtaining a feature line mask. This mask is then converted into feature lines in the agricultural machinery coordinate system. The number of feature lines within the wheelbase range of the agricultural machinery is counted to determine the region width. This region width is used for region division, and the feature lines in the agricultural machinery coordinate system are updated according to the region division results and inter-frame correspondence, increasing the stability of the feature lines processed each time and adapting in real-time to complex agricultural machinery operation scenarios. When the distance to the agricultural machinery... When a feature line exists in the area closest to the midpoint of the front axle, or when feature lines exist in the symmetrical area of the midpoint of the front axle, the feature line is added to the feature line list. Based on the feature line list, the navigation line is determined, taking into full account situations such as missed detections, false detections, and line breaks, greatly improving the stability of navigation line generation. Finally, based on the navigation line and combined with the agricultural machinery's direction line, the heading error and lateral error are obtained for agricultural machinery navigation. This improves the efficiency and stability of information acquisition in agricultural machinery automatic driving scenarios, solves the information stability problem when using a single sensor to generate navigation signals, reduces jitter, and greatly expands the application scope of visual navigation methods. Attached Figure Description
[0059] Figure 1 This is a schematic diagram of the overall process of a vision-based stable navigation signal generation method provided as an exemplary embodiment of this application.
[0060] Figure 2 This is a schematic diagram illustrating the number of feature lines within the wheel track range of a vision-based stable navigation signal generation method, which is provided as another exemplary embodiment of this application.
[0061] Figure 3 This is a schematic diagram of region division for a vision-based stable navigation signal generation method, which is provided as another exemplary embodiment of this application.
[0062] Figure 4 This is a schematic diagram of feature line offset for a vision-based stable navigation signal generation method provided as an exemplary embodiment of this application. Detailed Implementation
[0063] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.
[0064] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0065] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.
[0066] To address the contradiction between the low stability of navigation signal generation using a single sensor and the high cost of generating navigation signals using multiple sensors, this application provides a vision-based stable navigation signal generation method. Based on pure visual input signals, it can generate stable navigation signals in real time, providing heading and lateral error information for autonomous driving. This method can be applied to the field of agricultural machinery autonomous driving, greatly expanding the scope of application of visual navigation methods.
[0067] This application provides a vision-based stable navigation signal generation method, the main steps of which are described below.
[0068] Acquire image information from a color camera;
[0069] The image information is identified using visual target recognition technology to obtain a feature row mask;
[0070] The feature row mask is converted into feature lines in the agricultural machinery coordinate system. The number of feature lines within the wheel track range of the agricultural machinery is counted to determine the region width. The region width is then used to divide the region.
[0071] Update the feature lines in the agricultural machinery coordinate system according to the region division results and the inter-frame correspondence;
[0072] When a feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery, or when a feature line exists in the symmetrical area of the midpoint of the front axle of the agricultural machinery, the feature line is added to the feature line list.
[0073] Based on the list of feature lines, determine the navigation lines;
[0074] Based on the navigation lines and combined with the agricultural machinery direction lines, the heading error and lateral error are obtained and used for agricultural machinery navigation.
[0075] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.
[0076] Reference Figure 1 The system identifies the first frame of image information captured by the color camera and obtains the feature row mask. Using visual target recognition technologies, such as vegetation coefficients and deep learning segmentation models, the feature row mask is identified. The feature row mask can be crop rows, field ridge rows, or plastic film mulch rows, etc.
[0077] Obtain the feature lines corresponding to the feature row mask. For each feature row mask, obtain the midpoint coordinates (X, Y, F, G) of each row in the image coordinate system. i Y i Each feature row mask can obtain n (X) i Y i ), where n is the total number of rows within the crop coverage area of the agricultural machinery in the image coordinate system. For these n points, linear fitting techniques, such as RANSAC and least squares, are used to transform the midpoint coordinates of the n feature row masks into feature lines.
[0078] The feature lines are transformed from the pixel coordinate system to the agricultural machinery coordinate system. Using pre-acquired agricultural machinery and camera parameters, including: wheelbase (if the front and rear wheelbases are different, the larger one is used), front and rear axle distance, camera mounting angle, camera mounting height, distance from the camera to the midpoint of the front axle, distance from the camera to the central axis, and camera intrinsic parameters, each feature line is transformed from the pixel coordinate system to the agricultural machinery coordinate system (x...). i y i The default feature line height is 0.
[0079] Next, the number of feature lines within the wheelbase range of the agricultural machinery is obtained. Based on the number of feature lines, the region width is determined, and the region is divided and initialized. By analyzing the feature lines in the agricultural machinery coordinate system, the number of feature lines within the wheelbase AM is statistically analyzed. w Feature lines within the range l num The number of wheel tracks. The wheel track of the agricultural machinery can be between 1.0 and 3.0 meters, and the number of characteristic lines it crosses should be no less than one line and no more than ten lines. In this embodiment, the wheel track AM of the agricultural machinery... w Take 1.5m.
[0080] Reference Figure 2 Agricultural machinery wheel track AM w Feature lines within the range l num As shown by the green line. The red dot represents the midpoint of the agricultural machinery's wheel track, and the area will be divided to the left and right from this center.
[0081] Calculate the width r of the region w ,include,
[0082] r w =AM W / l num
[0083] Reference Figure 3 Based on the area width, the symmetrical range to the left and right of the midpoint of the wheel track of the agricultural machinery is divided into multiple areas.
[0084] When the number of feature lines within the range of the agricultural machinery wheel track is odd, that is, for the odd number scenario, the region closest to the midpoint of the agricultural machinery wheel track is matched with region ID 0. With the region width as the region interval, the region ID is matched in the order of left negative and right positive, from near to far, ±1, ±2.
[0085] When the number of feature lines within the wheel track range of the agricultural machinery is even, that is, for even-number scenarios, it is stipulated that there is no region with ID 0. With the region width as the region interval, the symmetrical regions at the distance from the midpoint of the front axle of the agricultural machinery are matched successively at ±1 and ±2.
[0086] Two symmetrical area IDs are added outside the wheel track range of the agricultural machinery. By adding an extra set of ID areas outside the wheel track range of the agricultural machinery, such as the ±2 area in the figure, the situation where the outer side of the agricultural machinery sways and runs over the feature row is reduced.
[0087] Determine whether the region division is complete.
[0088] When the region division is not completed, continue to obtain the number of feature lines within the range of the agricultural machinery wheel track, determine the region width based on the number of feature lines, perform region division and initialize the divided regions, until the region division is completed.
[0089] If a feature line is detected in a region, a filtering method is used to update the feature line. Based on the region ID corresponding to each region, the region ID corresponding to the feature line within that region is determined. Utilizing the correspondence of the same ID across different frames, and combining this with filtering methods such as mean filtering and low-pass filtering, the feature line is updated, thereby increasing the stability of the feature line itself in each processing iteration. The feature line update formula is as follows:
[0090]
[0091] line ID (x) and line ID (y) represents the x-axis and y-axis coordinates of the feature line within the corresponding region ID, respectively. i and y i The x-axis and y-axis coordinates of the feature line calculated for the corresponding previous i frames, where i takes values from [-n, 0], representing the distance from the previous n frames to the current frame 0. i The weights can be set according to the filter used. For example, for mean filtering,
[0092] By utilizing inter-frame information in video to update the position information of feature lines, stable, real-time updated feature lines can be obtained, which helps improve the stability of subsequently generated navigation signals. The update process can be combined with filtering, such as median filtering or low-pass filtering.
[0093] After the region division is completed, the final candidate crop line regions are shown in the green circles in the figure. Determining the region width based on the number of feature lines and dividing the region not only improves the adaptability of agricultural machinery when dividing navigation regions in complex operation scenarios, which is conducive to expanding the application scope of autonomous driving, but also allows for the tracking and processing of feature lines based on region ID labels, thereby utilizing multi-frame information to update feature lines within the region.
[0094] Once the region is divided, determine whether feature lines have been detected in the region to construct a feature line list.
[0095] When a feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery, or when a feature line exists in the symmetrical area of the midpoint of the front axle of the agricultural machinery, the feature line is added to the feature line list.
[0096] Specifically, determine whether characteristic lines exist in both the left and right symmetrical regions.
[0097] For example, assuming there is a feature line with region ID -1 on the left, we continue to check if there is a feature line with region ID 1 on the right. If so, we determine that feature lines exist in both symmetrical regions, and add both feature lines from the left and right regions to the feature line list V. line .
[0098] For odd-numbered scenarios, if the zero-region feature line exists, add the zero-region feature line to the feature line list V as well. line .
[0099] In one embodiment, when there is no feature line in the area closest to the midpoint of the front axle of the agricultural machinery or when there is no matching feature line in the symmetrical area of the midpoint of the front axle of the agricultural machinery, the feature line in the agricultural machinery coordinate system is offset.
[0100] Add the feature lines in the offset agricultural machinery coordinate system that meet the conditions to the feature line list.
[0101] Reference Figure 4 Specifically, feature lines that do not meet the conditions are moved to the minimum ID region, region 0, or region ±1 based on their current location. The formula for calculating the movement offset is:
[0102] off l =(-1*min ID -line ID )*r w
[0103] off r =(min ID -line ID )*r w
[0104] In the formula, for odd-numbered scenarios, min ID =0; for even-number scenarios, min ID =1. line ID This corresponds to the region ID of the feature line. w This represents the width of the region.
[0105] For the feature line on the left side of the agricultural machinery, the offset feature line is calculated as follows:
[0106] line′[[x′0,y′0],[x′1,y′1]]=line[[x0+off l ,y0],[x1+off l ,y1]]
[0107] For the feature line on the right side of the agricultural machinery, the offset feature line is calculated as follows:
[0108] line′[[x′0,y′0],[x′1,y′1]]=line[[x0+off r ,y0],[x1+off r ,y1]]
[0109] Add the moved feature line to the feature line list V. line It fully considers scenarios such as missed detection, false detection, and line breakage, greatly improving the stability of the generated navigation signals.
[0110] Next, based on the list of feature lines, the navigation lines are determined, including:
[0111]
[0112] In the formula, navi line For navigation lines, len(V) is the total number of feature lines in the feature line list, and V line (i) is any feature line in the feature line list.
[0113] Based on the navigation lines and combined with the agricultural machinery direction lines, the heading error and lateral error are obtained and used for agricultural machinery navigation.
[0114] Specifically, after obtaining the navigation lines, they can be combined with the agricultural machinery direction lines (AGR). lineThe heading and lateral errors are calculated using [[0, 0], [0, -wheelbase]]. The navigation line and the agricultural machinery heading line are obtained based on the agricultural machinery coordinate system, which takes the midpoint of the front axle of the agricultural machinery as its origin. Therefore, the agricultural machinery heading line is agr. line [[0,0],[0,-wheelbase]], where wheelbase is the distance between the front and rear axles of the agricultural machinery, and [0,0] and [0,-wheelbase] correspond to the midpoint of the front axle and the midpoint of the rear axle, respectively.
[0115] During the calculation, the navigation lines and agricultural machinery direction lines are first converted into vector form. The calculation formula is as follows:
[0116]
[0117] Calculate vectors The dot product and cross product include,
[0118] dot = a x *n x +a y *n y
[0119] cross = a x *n y -a y *n x
[0120] The yaw angle, i.e., the heading error, is calculated using the dot product (dot) and cross product (cross). The calculation formula is as follows:
[0121] θ = arctan2(cross, dot)
[0122]
[0123] Lateral error is divided into front axle midpoint lateral error d front Lateral error d at the midpoint of the rear axle rear Based on the navigation line, the front axle midpoint [0,0], and the rear axle midpoint [0,-wheelbase], the lateral error of the front axle midpoint / the lateral error of the rear axle midpoint is the perpendicular distance relative to the navigation line. The formulas for calculating the lateral errors of the front and rear axle midpoints are as follows:
[0124]
[0125] The final heading error and lateral error are used for agricultural machinery navigation.
[0126] Furthermore, the heading error and lateral error can be filtered to generate the final navigation information.
[0127] The navigation lines are filtered based on the previous navigation lines.
[0128] Based on the previous heading error and the previous lateral error, the heading error and the lateral error are filtered.
[0129] The filtering operation is a prior art in this field and will not be described in detail here.
[0130] By performing filtering operations, the final output heading error and lateral error are made more stable.
[0131] In one embodiment, if no feature line is detected in a region, it is determined whether the feature line in that region has expired.
[0132] Specifically, when no feature lines are detected for M consecutive frames, it is determined that there are no feature lines in the region, and the feature lines in the region are deleted.
[0133] When a feature line is detected in M consecutive frames, the feature line of the previous frame is used as the feature line of the region, i.e., the feature line is preserved.
[0134] Where M is a positive integer.
[0135] If no feature lines are found in the new frame, check if no feature lines have been detected in the region for a certain period of time. If so, clear the feature lines in the region; otherwise, use the previous feature lines.
[0136] By adding some delayed disappearance mechanisms: if no feature line is detected in a region for M consecutive frames, the feature line in that region is deleted; otherwise, the feature line of the previous frame is used, in order to reduce the jitter of the input signal caused by accidental missed detection.
[0137] In summary, a vision-based stable navigation signal generation method acquires image information solely from a color camera, requiring no additional equipment such as LiDAR, depth cameras, inertial navigation systems, or positioning devices, resulting in low equipment costs. It employs visual target recognition technology to identify image information, obtaining feature line masks. These masks are then converted into feature lines in the agricultural machinery coordinate system. The number of feature lines within the wheelbase range of the agricultural machinery is counted to determine the region width. This region width is used for region division, and the feature lines in the agricultural machinery coordinate system are updated according to the region division results and inter-frame correspondence. This increases the stability of the feature lines processed each time, allowing for real-time adaptation to agricultural machinery operations. In complex scenarios, when a feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery, or when feature lines exist in the symmetrical areas of the midpoint of the front axle, the feature line is added to the feature line list. Based on the feature line list, the navigation line is determined, fully considering situations such as missed detections, false detections, and line breaks, greatly improving the stability of navigation line generation. Finally, based on the navigation line and combined with the agricultural machinery's direction line, the heading error and lateral error are obtained for agricultural machinery navigation. This improves the efficiency and stability of information acquisition in agricultural machinery automatic driving scenarios, solves the information stability problem when using a single sensor to generate navigation signals, reduces jitter, and greatly expands the application scope of visual navigation methods.
[0138] A vision-based stable navigation signal generation method uses only a color camera to collect image information of agricultural machinery operation scenes, which significantly reduces equipment costs. Users can intuitively understand the meaning of the information when using it.
[0139] A vision-based stable navigation signal generation method generates navigation control information based on pure vision technology. On the one hand, it significantly reduces costs compared to multi-sensor fusion methods; on the other hand, it greatly increases stability compared to existing pure vision navigation methods, thereby expanding the application scope of pure vision navigation methods in agricultural machinery automatic driving scenarios.
[0140] A vision-based stable navigation signal generation method uses only data collected by a camera as input, and combines target recognition, line fitting, coordinate system transformation, tracking, filtering and other techniques to generate stable navigation control information, thus solving the instability problem of navigation signals generated based on pure visual information.
[0141] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0142] This application also provides a vision-based stable navigation signal generation device, which corresponds one-to-one with the vision-based stable navigation signal generation method described in the above embodiments. The vision-based stable navigation signal generation device includes...
[0143] An image module is used to acquire image information from a color camera;
[0144] The masking module is used to identify the image information using visual target recognition technology to obtain a feature row mask;
[0145] The partitioning module is used to convert the feature row mask into feature lines in the agricultural machinery coordinate system, count the number of feature lines within the wheel track range of the agricultural machinery, determine the region width, and use the region width to partition the region.
[0146] The update module is used to update the feature lines in the agricultural machinery coordinate system according to the region division results and the inter-frame correspondence.
[0147] The collection module is used to add the feature line to the feature line list when the feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery or when the feature line exists in the symmetrical area of the midpoint of the front axle of the agricultural machinery.
[0148] The navigation line module is used to determine navigation lines based on the feature line list;
[0149] The navigation signal module is used to obtain heading error and lateral error based on the navigation line and the agricultural machinery direction line for agricultural machinery navigation.
[0150] For specific limitations regarding a vision-based stable navigation signal generation device, please refer to the limitations of a vision-based stable navigation signal generation method mentioned above, which will not be repeated here.
[0151] The modules in the aforementioned vision-based stable navigation signal generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0152] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements any of the aforementioned vision-based stable navigation signal generation methods.
[0153] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described vision-based stable navigation signal generation methods.
[0154] In one embodiment, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the above-described vision-based stable navigation signal generation methods.
[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. When executed, the computer program may include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0156] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.
Claims
1. A vision-based method for generating stable navigation signals, characterized in that, Includes the following steps, Acquire image information from a color camera; The image information is identified using visual target recognition technology to obtain a feature row mask; The feature row mask is converted into feature lines in the agricultural machinery coordinate system. The number of feature lines within the wheel track range of the agricultural machinery is counted to determine the region width. The region width is then used to divide the region. Update the feature lines in the agricultural machinery coordinate system according to the region division results and the inter-frame correspondence; When a feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery, or when a feature line exists in the symmetrical area of the midpoint of the front axle of the agricultural machinery, the feature line is added to the feature line list. Based on the list of feature lines, determine the navigation lines; Based on the navigation lines and combined with the agricultural machinery direction lines, the heading error and lateral error are obtained and used for agricultural machinery navigation.
2. The vision-based stable navigation signal generation method according to claim 1, characterized in that, It also includes the following steps, When there is no feature line in the area closest to the midpoint of the front axle of the agricultural machinery, or when there is no matching feature line in the symmetrical area of the midpoint of the front axle of the agricultural machinery, the feature line in the agricultural machinery coordinate system is offset. Add the feature lines in the offset agricultural machinery coordinate system that meet the conditions to the feature line list.
3. The vision-based stable navigation signal generation method according to claim 1, characterized in that, The step of dividing the region using the region width includes: When the number of feature lines within the wheel track range of the agricultural machinery is even, the symmetrical regions at the distance from the midpoint of the front axle of the agricultural machinery are matched with region IDs of ±1, ±2, ±3, ..., ±N, where N is a positive integer, based on the region interval of the region width. When the number of feature lines within the wheel track range of the agricultural machinery is odd, the region ID of the region closest to the midpoint of the front axle of the agricultural machinery is initialized to 0, and the symmetrical regions with region ID 0 are successively ±1, ±2, ±3, ..., ±N, where N is a positive integer; Add two symmetrical region IDs outside the wheel track range of agricultural machinery.
4. The vision-based stable navigation signal generation method according to claim 1, characterized in that, The step of determining the navigation lines based on the feature line list includes: In the formula, navi line For navigation lines, len(V) is the total number of feature lines in the feature line list, and V line (i) is any feature line in the feature line list.
5. The vision-based stable navigation signal generation method according to claim 1, characterized in that, The step of converting the feature row mask into feature lines in the agricultural machinery coordinate system includes: Obtain the midpoint coordinates of each row of the feature row mask in the image coordinate system; Based on the coordinates of each midpoint, and combined with straight line fitting technology, the feature row mask is converted into feature lines; Based on the preset agricultural machinery parameters and camera parameters, the feature lines are transformed into the agricultural machinery coordinate system.
6. The vision-based stable navigation signal generation method according to any one of claims 1-5, characterized in that, It also includes the following steps, If no feature lines are detected for M consecutive frames, it is determined that there are no feature lines in the region. When a feature line is detected in M consecutive frames, the feature line of the previous frame is used as the feature line of the region. Where M is a positive integer.
7. The vision-based stable navigation signal generation method according to claim 6, characterized in that, After determining the navigation lines based on the feature line list, the method further includes: The navigation lines are filtered based on the previous navigation lines.
8. The vision-based stable navigation signal generation method according to claim 6, characterized in that, After the step of obtaining the heading error and lateral error based on the navigation line and the agricultural machinery direction line for agricultural machinery navigation, the method further includes: Based on the previous heading error and the previous lateral error, the heading error and the lateral error are filtered.
9. A vision-based stable navigation signal generation device, characterized in that, include, An image module is used to acquire image information from a color camera; The masking module is used to identify the image information using visual target recognition technology to obtain a feature row mask; The partitioning module is used to convert the feature row mask into feature lines in the agricultural machinery coordinate system, count the number of feature lines within the wheel track range of the agricultural machinery, determine the region width, and use the region width to partition the region. The update module is used to update the feature lines in the agricultural machinery coordinate system according to the region division results and the inter-frame correspondence. The collection module is used to add the feature line to the feature line list when the feature line exists in the area closest to the midpoint of the front axle of the agricultural machinery or when the feature line exists in the symmetrical area of the midpoint of the front axle of the agricultural machinery. The navigation line module is used to determine navigation lines based on the feature line list; The navigation signal module is used to obtain heading error and lateral error based on the navigation line and the agricultural machinery direction line for agricultural machinery navigation.
10. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.
12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.