Navigation path planning method and device, electronic equipment and readable storage medium

Abstract

Embodiments of the present application provide a navigation path planning method and device, electronic equipment and readable storage medium, and relate to the technical field of navigation. The method comprises: obtaining a local image of a first target shooting image based on a central axis of a work equipment; converting the local image to obtain a target bird's-eye view; performing crop identification on the target bird's-eye view to determine at least one crop row area; and obtaining a navigation path according to a center line of the at least one crop row area and a preset scale factor, wherein the preset scale factor is used to represent the relationship between the distance between target bird's-eye view pixels and the actual distance. In this way, the monocular vision crop actual positioning problem is effectively solved based on a monocular vision image, so as to obtain an actual navigation path that can be used for work on crops; and the interference of other crop rows on navigation can be excluded through view interception, while reducing the data processing amount.

Description

Technical Field

[0001] This application relates to the field of navigation technology, and more specifically, to a navigation path planning method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Automated navigation for crops can improve operational efficiency and reduce costs. Currently, navigation is generally based on multi-sensor fusion technology or pure vision-based navigation. However, the actual navigation accuracy of multi-sensor fusion technology and monocular vision is relatively low; binocular vision navigation technology has high hardware requirements for image acquisition equipment and complex algorithms. Therefore, how to improve the accuracy of real-time navigation without increasing hardware costs has become a pressing technical problem for those skilled in the art. Summary of the Invention

[0003] This application provides a navigation path planning method, apparatus, electronic device, and readable storage medium, which can effectively solve the problem of actual crop positioning based on monocular vision images, thereby obtaining an actual navigation path that can be used for crop operations, improving navigation accuracy, and obtaining the path quickly.

[0004] The embodiments of this application can be implemented as follows: In a first aspect, embodiments of this application provide a navigation path planning method, the method comprising: Based on the central axis of the working equipment, a partial image of the first target image is obtained; The local image is transformed to obtain a bird's-eye view of the target; Crop identification is performed on the target bird's-eye view to determine at least one crop row area; A navigation path is obtained based on the centerline of the at least one crop row area and a preset scale factor, wherein the preset scale factor is used to represent the relationship between the distance between the pixels of the target bird's-eye view and the actual distance.

[0005] Secondly, embodiments of this application provide a navigation path planning device, the device comprising: The cropping module is used to obtain a partial image of the first target image based on the central axis of the working equipment; A conversion module is used to convert the local image to obtain a bird's-eye view of the target; The identification module is used to identify crops in the target bird's-eye view and determine at least one crop row area; The path acquisition module is used to obtain a navigation path based on the centerline of the at least one crop row area and a preset scale factor, wherein the preset scale factor is used to represent the relationship between the distance between the pixels of the target bird's-eye view and the actual distance.

[0006] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the navigation path planning method described in the foregoing embodiments.

[0007] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the navigation path planning method as described in the foregoing embodiments.

[0008] The navigation path planning method, apparatus, electronic device, and readable storage medium provided in this application, upon obtaining a first target image, obtains a target bird's-eye view corresponding to a local image in the first target image by view cropping and field-of-view transformation based on the central axis of the working equipment. By identifying crops in the target bird's-eye view, at least one crop row region is obtained. Then, based on the centerline of the at least one crop row region and a preset scale factor, a navigation path is obtained. The preset scale factor represents the relationship between the distance between pixels in the target bird's-eye view and the actual distance. Thus, by cropping the field of view based on the central axis of the working equipment, while ensuring the acquisition of a local image for planning the navigation path, interference from other crop rows on navigation is reduced, and the amount of data processing is reduced, thereby improving the path acquisition speed. Furthermore, by transforming the local image into a bird's-eye view and using the preset scale factor, the actual positioning problem of crops in monocular vision can be effectively solved based on monocular vision images, thereby obtaining an actual navigation path that can be used for crop operations and improving navigation accuracy. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application; Figure 2 One of the flowcharts of the navigation path planning method provided in the embodiments of this application; Figure 3 A schematic diagram illustrating the acquisition of navigation paths provided in the embodiments of this application; Figure 4 This is a schematic diagram of the centerline calibration of the working equipment provided in the embodiments of this application; Figure 5 This is a schematic diagram of field calibration provided for an embodiment of this application; Figure 6 for Figure 2 One of the flowcharts illustrating the sub-steps included in step S140; Figure 7 A second flowchart illustrating the navigation path planning method provided in this application embodiment; Figure 8 for Figure 7 The second flowchart of the sub-steps included in step S140; Figure 9 The third flowchart illustrating the navigation path planning method provided in this application embodiment; Figure 10 This is a block diagram of a navigation path planning device provided in an embodiment of this application.

[0011] Icons: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication unit; 200 - Navigation path planning device; 210 - Interception module; 220 - Conversion module; 230 - Identification module; 240 - Path acquisition module. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0013] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0014] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0015] Currently, visual crop navigation technology can be broadly divided into multi-sensor fusion technology and pure visual navigation technology.

[0016] In multi-sensor fusion technology, a satellite navigation system is used for coarse heading adjustment, while a visual navigation system is used for fine heading adjustment, ensuring the object moves along the centerline of the crop row. Specifically, the satellite navigation system makes low-frequency, large-amplitude adjustments to the heading, while the visual navigation system makes high-frequency, small-amplitude adjustments. However, current multi-sensor fusion positioning and navigation technologies primarily rely on loose coupling. Other sensors, such as GNSS systems and RTK, mainly constrain drastic angle changes, providing some assistance but offering little improvement in positioning and navigation accuracy, resulting in relatively poor navigation precision.

[0017] For pure vision monocular navigation technology, edge detection and predetermined information about crop size and the distance between each crop can be used to detect plants in an image, thereby obtaining the plant rows in the image. Then, navigation is performed by controlling the angular direction based on the angle of the plant rows. Navigation can also be achieved using pure vision binocular navigation technology.

[0018] For pure vision-based sensor navigation, binocular technology suffers from disadvantages such as high equipment requirements and complex algorithms. Monocular vision technology can only obtain angular information and cannot accurately locate plants, resulting in poor navigation accuracy.

[0019] In view of the above situation, this application provides a navigation path planning method, device, electronic device and readable storage medium based on monocular vision images. The area to be navigated is extracted based on the central axis of the working equipment, and the specific positioning coordinates of the crops can be obtained by combining a preset scale factor, thereby obtaining the specific positioning coordinates of the navigation path. Based on the navigation path, the angle and position can be accurately controlled, thus improving navigation accuracy. Furthermore, by extracting the area, interference from other crop rows on navigation can be avoided, while reducing the amount of data processing.

[0020] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0021] Please refer to Figure 1 , Figure 1 This is a block diagram of an electronic device 100 provided in an embodiment of this application. The electronic device 100 may be part of a work device, or it may be a device independent of the work device but installed on it. The electronic device 100 may be communicatively connected to a camera for planning navigation paths based on the acquired images.

[0022] The electronic device 100 may include a memory 110, a processor 120, and a communication unit 130. The memory 110, processor 120, and communication unit 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0023] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0024] The processor 120 is used to read / write data or programs stored in the memory 110 and perform corresponding functions. For example, the memory 110 stores a navigation path planning device 200, which includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 executes various functional applications and data processing by running the software programs and modules stored in the memory 110, such as the navigation path planning device 200 in this embodiment, thereby implementing the navigation path planning method in this embodiment.

[0025] The communication unit 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0026] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0027] Please refer to Figure 2 , Figure 2 This is one of the flowcharts illustrating the navigation path planning method provided in this application embodiment. The method can be applied to the aforementioned electronic device 100. The specific flow of the navigation path planning method is described in detail below. In this embodiment, the method may include steps S110 to S140.

[0028] Step S110: Based on the central axis of the working equipment, obtain a partial image of the first target image.

[0029] In this embodiment, the operating equipment is used for operating crops. The crops can be food crops or cash crops. The first target image can be an image captured by a vision sensor at certain time intervals or from a certain distance; it can also be an image extracted from video obtained using a vision sensor. The specific image acquisition method can be set according to actual needs. Image capture can be performed while the operating equipment is in motion to obtain the first target image.

[0030] A partial image for path planning can be extracted from the first target image based on the centerline of the working equipment. That is, the actual area corresponding to the partial image includes the area requiring navigation, but is smaller than the actual area corresponding to the first target image. This reduces data processing volume and facilitates obtaining a path related to the centerline based on the partial image, thereby enabling the driving control of the working equipment.

[0031] Step S120: Convert the local image to obtain a bird's-eye view of the target.

[0032] The local image can be converted to a bird's-eye view using a pre-set transformation matrix or other field-of-view transformation algorithm to obtain the target bird's-eye view.

[0033] Step S130: Perform crop identification on the target bird's-eye view to determine at least one crop row area.

[0034] Given the target bird's-eye view, crop rows can be segmented and extracted using adaptive threshold segmentation or AI semantic segmentation to determine at least one crop row region. A crop row region represents the area containing a single row of crops. In cases of dense crop growth, such as corn, a crop row region is a connected region; in cases of sparse crop growth, such as fruit trees, a crop region may include multiple disconnected crop regions. For example, ... Figure 3 As shown, on both sides of the first navigation center line, there are two crop row areas, each of which includes the area where two crops are located.

[0035] The plants in different crop rows can be the same or different. For example, different crop rows may contain the same fruit trees or other crops; or some crop rows may contain fruit trees or other crops, while others may contain windbreaks; or some crop rows may contain fruit trees, while others may contain corn, etc. Understandably, the specific types of plants in the crop rows are determined by the actual environment where the equipment is located. For example, one side of the equipment may be a windbreak, a plot boundary, or fruit trees, while the other side may be crops; or both sides of the equipment may be crops.

[0036] Step S140: Obtain a navigation path based on the centerline of the at least one crop row area and a preset scale factor.

[0037] Once the crop row area is determined, the centerline of one crop row area can be offset according to the actual situation to obtain the navigation line required by the operating equipment; alternatively, the centerline of the total area composed of multiple crop row areas can be used as the navigation line required by the operating equipment. Then, based on a preset scale factor, the coordinates of each point on the navigation line in the target bird's-eye view are transformed to obtain the specific positioning coordinates corresponding to each point on the navigation line. In this way, a navigation path including specific positioning coordinates can be obtained.

[0038] The center line of a crop row area, also known as the crop row center line, is used to represent a row of crops in a target bird's-eye view. For example... Figure 3 As shown, for the crop row area located to the left of the first navigation center line, the black dashed line is the center line of the crop rows in that area, representing the crops in that area. The center line of the total area composed of multiple crop row areas can be a center line that divides the total area into two equal parts. For example... Figure 3As shown, the center line between the crop row areas on the left and right sides can be used as the center line of the total area formed by these two crop row areas. The preset scale factor is used to represent the relationship between the distance between pixels in the target bird's-eye view and the actual distance.

[0039] This navigation path can be used in scenarios where one side of the work equipment is a windbreak / plot boundary / fruit trees, and the other side is crops. It can also be used in scenarios where both sides of the work equipment are crops. The specific navigation application scenario can be determined based on the specific use case. For example, the navigation path obtained by the above method can be used for navigation between fruit trees.

[0040] In this way, by extracting the area to be navigated based on the central axis of the operating equipment and combining it with a preset scale factor, the specific positioning coordinates of the crops can be obtained, and then the specific positioning coordinates of the navigation path can be obtained. Based on this navigation path, the angle and position can be accurately controlled, thus improving navigation accuracy. Furthermore, by extracting the area, interference from other crop rows on navigation can be avoided, while reducing the amount of data processing.

[0041] Optionally, as a possible implementation, the field of view can be calibrated in advance to determine the parameters required for the screenshot and the preset scale factor. The calibration process is described below.

[0042] A vision sensor can be installed on the work equipment to ensure that an image can be obtained directly in front of the work equipment. The specific installation position and angle can be set according to actual needs and are not specifically limited here.

[0043] The central axis of the working equipment can be measured in the following way: Assemble a circular plastic pipe as shown in the diagram. Figure 4 The bracket shown includes mutually perpendicular sides AO and BC. Position side BC close to the front side of the rear wheel of the work equipment, making it as parallel as possible to the rear wheel axle. Since the lengths of sides BO and CO are equal on the bracket structure, the operator uses a tape measure to measure the lengths of point B to the outer wall of the left wheel and point C to the outer wall of the right wheel. When these lengths are equal, the direction of side OA is assumed to be the forward path direction (i.e., the direction of the work equipment's central axis, or the direction of the work equipment's centerline). If they are unequal, they must be adjusted to be equal.

[0044] Then, the range and size of the field of view can be determined based on the established center flight path direction. Optionally, the range, size, and location of the field of view can be defined first. For example, assuming the field of view is 2×2m and located 1m in front of the working equipment, the field of view, calibrated with the center flight path direction as a reference, can be defined as follows: Figure 5 As shown, the field of view obtained at this time is a rectangle with Ly as the length and Lx as the width.

[0045] Optionally, the size and location of the field of view can be determined by considering the number and length of navigation paths required based on that field of view. For example, if only one navigation path is needed based on that field of view, the size of the field of view can be limited to include only two rows of crops, such as... Figure 3 and Figure 5 As shown, the field of view is defined by using the central axis of the equipment as the center line, and selecting a rectangular area containing two rows of crops on either side of the center line. Similarly, when multiple navigation paths need to be obtained, the field of view can be limited to include two or more rows of crops. Again, using the central axis of the equipment as the center line, the field of view is defined by selecting an area containing two or more rows of crops outside the center line. The corner positions of the field of view will also be adjusted accordingly.

[0046] In the above, it should be noted that, firstly, when only a navigation path needs to be obtained based on this field of view, since the working equipment may not necessarily be located between two rows of crops, for example, there may only be crops on one side of the working equipment, while the other side may be, but is not limited to, the boundary of the plot or a windbreak. In this case, the selection of the field of view size is essentially the same as the principle described above, that is, the central axis of the working equipment is used as the center line of the field of view, and a rectangular area including the areas on both sides of the center line of the field of view (such as the area of ​​one row of crops and the area of ​​the plot boundary) is selected as the field of view range. The field of view range does not necessarily have to be a rectangular area and can be adjusted according to actual needs. Secondly, although the above description is based on selecting the field of view size based on crop rows, in another implementation, the relationship between the number of navigation paths and the width of the field of view can be determined in advance based on the required number of navigation paths and the width of the crop rows. For example, assuming the width of one crop row is d1 meters, the spacing between crop rows is d2 meters, and the number of navigation paths is 2, then at least 3 crop rows are needed to obtain 2 navigation paths. Therefore, the width of the field of view can be 2d2 + 3d1. However, when the field of view is formed by extending the same distance to both sides of the central axis of the working equipment, considering symmetry and the integrity of the included crop rows, the width of the field of view can be adjusted to 3d2 + 4d1. Therefore, the relationship between the number of navigation paths n and the width w of the field of view can be expressed as: w = nd2 + (n + 1)d1.

[0047] In addition, the length of the field of view can be set as needed. For example, it can be set to be the same as the width of the field of view, or it can be larger or smaller than the width of the field of view, as long as the required navigation path can be obtained.

[0048] After determining the field of view, the coordinates of the four corner points in the image captured by the vision sensor can be extracted as calibration parameters. Based on these corner calibration parameters, a portion of the image captured by the vision sensor can be cropped and converted into a bird's-eye view. Then, by combining the number of horizontal and vertical pixels in the bird's-eye view with the actual size of the field of view, the scale factor (i.e., the preset scale factor) of the field of view region can be obtained. The X-direction scale factor is: Y-direction scaling factor: , Maximum horizontal pixel distance in bird's-eye view This indicates the maximum vertical pixel distance in the bird's-eye view. , This indicates the actual size of the field of view.

[0049] The image coordinates of the four corner points can be used as preset image coordinates; alternatively, the image coordinates of one corner point can be used as preset image coordinates, and the corresponding size and position information can be set to extract the image of the field of view from the captured image.

[0050] After calibration, when navigation path planning is needed, a partial image can be obtained by cropping a view from the first target image based on the preset image coordinates. Then, a field-of-view transformation is performed on this partial image to obtain a bird's-eye view of the target. Figure 3 As shown, the center line of the field of view of the target in the bird's-eye view can overlap with the extension line of the central axis of the working equipment.

[0051] After obtaining the target bird's-eye view, one possible implementation is to perform adaptive threshold segmentation on the target bird's-eye view to determine the crop row regions included in the target bird's-eye view. In this approach, different segmentation thresholds can be set; the target bird's-eye view is segmented into foreground and background using each segmentation threshold, and the variance between the foreground and background corresponding to each segmentation threshold is calculated for the segmentation results corresponding to each segmentation threshold; the maximum variance is selected, and the crop row regions included in the target bird's-eye view are obtained based on the segmentation result corresponding to the maximum variance.

[0052] As another possible implementation, a pre-defined model can be trained based on a dataset, and then used to identify crops in the target bird's-eye view to obtain crop regions. Each crop row region includes at least one crop region. If different crop regions are not connected, for example, adjacent fruit tree regions are not connected, the crop row regions can be determined by analyzing the positions of each crop region in the target bird's-eye view. In this way, the crop row regions corresponding to each row of crops can be obtained.

[0053] The dataset may include multiple sample images, along with corresponding crop and mulch film location information for each image. Crop and mulch film location information indicates the position of the crop area within the sample images, respectively. Based on the spatial relationship between crop planting and growth, a segmentation model with a large receptive field can be designed to analyze and infer crop morphological characteristics (such as color, leaf shape, and overall morphology) and spatial relationships (such as row distribution patterns) to segment the crop area. Simultaneously, by segmenting and identifying the mulch film area, and combining this with the positional relationship between the mulch film and the crop, both areas can be identified. Furthermore, the positional relationship between the mulch film and crop areas during segmentation can be used to further optimize the segmentation, resulting in more accurate crop area segmentation. This ensures the accuracy of the obtained crop row areas.

[0054] To ensure real-time navigation effectiveness, you can... Figure 6 As shown, a first navigation path is obtained based on the crop row area, so that the operating equipment can perform operations on the current crop row based on this first navigation path. Please refer to... Figure 6 , Figure 6 for Figure 2 One of the flowcharts illustrating the sub-steps included in step S140. In this approach, step S140 may include sub-steps S141 to S142. The navigation path includes the first navigation path.

[0055] Sub-step S141: Based on the center line of the field of view of the target bird's-eye view, determine the first planning range corresponding to the first navigation path from the target bird's-eye view.

[0056] When the target bird's-eye view can be used to determine at least one navigation path, the coordinates corresponding to the first area to be planned can be preset. Then, when the target bird's-eye view is obtained, the first area to be planned can be determined from the target bird's-eye view based on these coordinates. Alternatively, the relationship between the center line of the field of view of the target bird's-eye view and the first area to be planned can be obtained during field of view calibration, and the first area to be planned in the target bird's-eye view can be determined based on this relationship. It is understood that the above is only an example, and the first area to be planned can also be determined in other ways.

[0057] The size of the first planned area is determined based on the wheel track and crop row spacing of the operating equipment, or based on the wheel track, crop row width, and crop row spacing. When the operating equipment has two wheels, the wheel track represents the distance between the two center planes of the two wheels. When the operating equipment has a single wheel, the wheel track can be represented by 0. The crop row spacing represents the distance between adjacent crop rows.

[0058] First, it can be determined whether the wheel spacing is greater than the crop row spacing, and then the width of the first planned area can be determined based on the determination result. The following example illustrates how the width of the first planned area is determined. In this example, assume the width of one crop row is d1 meters, and the spacing between crop rows is d2 meters.

[0059] For example, when the wheel track of the working equipment is less than the crop row interval d2, it means that the working equipment can travel between adjacent crop rows. The width of the first planned area can be determined based on the crop row interval and the width of the two crop rows. For example, the width of the first planned area can be set to d2+2d1.

[0060] For example, when it is determined that the wheels of the working equipment need to travel on the outer side of two adjacent rows of crops based on the wheel track, crop row width d1, and crop row spacing d2, the width of the first planned area can be determined according to the crop row spacing and the width of the two crop rows. For example, the width of the first planned area can be set to d2+2d1.

[0061] For example, when it is determined that the wheels of the operating equipment need to travel on the outside of the middle row of three consecutive adjacent rows of crops based on the wheel track, crop row width d1, and crop row spacing d2, the width of the first planned area can be determined according to the spacing between two crop rows and the width of three crop rows. For example, the width of the first planned area can be set to 2d2 + 3d1.

[0062] Sub-step S142: Based on the centerline of the first crop row area included in the first planning range and the preset scale factor, the first navigation path is obtained.

[0063] Based on the location of each crop row area in the target bird's-eye view and the first area to be planned, the crop row areas included in the first area to be planned can be determined. The crop row areas included in the first area to be planned can be used as the first crop row area.

[0064] Optionally, as a possible implementation, the first crop row area included in the first planning range can be considered as a total region, the edge of which is determined by the edge of the first crop row area outside the first planning range. The centerline of this total region can be calculated, and the first navigation path can be obtained based on the centerline and a preset scale factor. In this way, the first navigation path can be obtained quickly.

[0065] As another possible implementation, in order to ensure the operational effectiveness based on the first navigation path, the first navigation path can be calculated based on the centerline (i.e., the centerline of the crop row) of each of the first crop row areas included in the first planning range and the preset scale factor.

[0066] In this method, when multiple first crop row areas are identified, if the number of first crop row areas is even, the two first crop row areas closest to or furthest from the center line of the first planned area can be designated as the first target crop row areas. The center line of the first planned area extends in the same direction as the first crop row areas. For example, if the first planned area includes four rows of crops, the two middle rows or the two outermost rows can be designated as the first target crop row areas.

[0067] When the number of first crop row regions is multiple and odd, the first crop row regions on either side of the first crop row region closest to the center line of the first planned area can be designated as the first target crop row regions. The number of crop row regions between each first target crop row region and the first crop row region closest to the center line of the first planned area is equal.

[0068] For example, if the first planned area includes 5 rows of crops, from left to right, they are A, B, C, D, and E. If row C is closest to the center line of the first planned area, then rows B and D on both sides of row C can be used as the first target crop row area, or rows A and E on both sides of row C can be used as the first target crop row area.

[0069] Then, for each first target crop row region, the center line of the crop row can be obtained through fitting. Optionally, the center of each crop can be obtained simultaneously when identifying the crops, and then, for the centers of crops located in the same row, methods such as linear or cubic curve fitting can be used to fit the centers of multiple crops to obtain the center line of each crop row. Alternatively, the center line of the crop row can be obtained by fitting based on the edges of each first target crop row region.

[0070] Next, the centerline between the centerlines of the crop rows in the two first target crop row areas is obtained as the first navigation centerline, and the first navigation path is obtained based on the first navigation centerline and the preset scale factor. The first navigation centerline is equidistant from the centerlines of the crop rows in the left and right first target crop row areas.

[0071] Optionally, the first navigation centerline can be converted into coordinates expressed as actual distance based on the preset size factor. Alternatively, a certain number of coordinates can be selected from the first navigation centerline according to control requirements, and then the selected coordinates can be converted into coordinates expressed as actual distance based on the preset size factor.

[0072] For example, such as Figure 3 As shown, path points R1 and R2 are selected from the first navigation centerline. Assuming the centerline of the first planned area overlaps with the extension of the centerline of the working equipment, a coordinate system can be established with the bottom right direction of the field of view as the positive direction of the X-axis and the forward path direction of the vehicle center (i.e., the center path direction) as the positive direction of the Y-axis. The number of pixels from path point R1 to the X and Y axes are R1 and R2, respectively. x R y The actual distance from path point R1 to the origin at the bottom center of the field of view can be calculated as follows: X direction: R x *f x ;Y direction: R y *f y Similarly, the actual positioning information of path point R2 can be calculated. In this way, the actual distance of each path point relative to the field of view can be calculated, and then converted to the coordinate system required for control planning.

[0073] When the first planning area includes only one row of crops, the first navigation path can be obtained based on at least one of the preset offset distance and the wheel track of the operating equipment, as well as the centerline of the first crop row area (i.e., the centerline of the crop row in the first crop row area). The preset offset distance can be set based on safety requirements and can be either a pixel distance or an actual distance.

[0074] Optionally, the first navigation path can be obtained solely based on the preset offset distance, the centerline of the first crop row area, and the preset scale factor.

[0075] For example, assuming that the only row of crops in the first planning range is located on the left, and the preset offset distance is a pixel distance, the center line of the first crop row area where the crop row is located can be offset to the right by a preset distance to obtain a navigation line, and then the first navigation path can be obtained based on the navigation line and the preset scale factor.

[0076] For example, assuming that the only row of crops in the first planning range is located on the left, and the preset offset distance is the actual distance, the initial navigation path can be obtained first based on the center line of the first crop row area where the crop is located and the preset scale factor. Then, the initial navigation path is offset to the right by a preset distance to obtain the first navigation path.

[0077] Alternatively, the first navigation path can be obtained solely based on the wheel track of the operating equipment, the centerline of the first crop row area, and a preset scale factor. For example, the actual path obtained based on the centerline of the first crop row area and the preset scale factor can be offset by a certain distance (this distance can be the wheel track or half of the wheel track, etc.) away from the first crop row area to obtain the first navigation path.

[0078] A first navigation path can also be obtained based on the preset offset distance, the wheelbase of the operating equipment, the centerline of the first crop row area, and the preset scale factor to ensure the safety of the operating equipment. For example, in this method, assuming that the only row of crops in the first planning area is located on the left, and the preset offset distance is the actual distance, an initial navigation path can be obtained based on the centerline of the first crop row area and the preset scale factor. Then, the path can be offset to the right by the preset offset distance and half the wheelbase of the operating equipment, and the path after the offset can be used as the first navigation path.

[0079] To reduce data processing while ensuring current navigation requirements are met, the calibrated field of view can include only one navigation path. In this case, the range corresponding to the target bird's-eye view can be a range used only to obtain the current navigation path. The current navigation path is used to guide the operating equipment to operate on the current crop row. When the target bird's-eye view includes only one row of crops, the current navigation path can be obtained based on the centerline of the crop row area, a preset offset rule, and the preset scale factor. The preset offset rule can be based on offsetting the centerline of the crop row area according to a preset offset distance and / or the wheelbase of the operating equipment.

[0080] When the target bird's-eye view includes multiple rows of crops, the current navigation path can be obtained based on the center lines of the multiple crop row areas corresponding to the multiple rows of crops and the preset scale factor.

[0081] In this embodiment, the specific description of how to obtain the navigation path when the range corresponding to the target bird's-eye view is only used to obtain the current navigation path can be referred to the description above on how to obtain the first navigation path corresponding to the first planned range, and will not be repeated here.

[0082] Once the first navigation path is obtained, navigation operations such as planning and controlling the vehicle can be performed by integrating information such as satellite positioning. For example, steering can be performed based on satellite positioning information, and the vehicle can be controlled to execute the first navigation path. It is understood that if navigation has not ended, steps S110 to S140 can continue to be executed until navigation ends.

[0083] To ensure operational efficiency, navigation paths to adjacent crop rows can be predicted in advance to further improve efficiency. Optionally, the navigation path may also include candidate navigation paths obtained based on the target bird's-eye view, where the candidate navigation paths are adjacent to the first navigation path and correspond to different crop rows. Please refer to... Figure 7 , Figure 7 This is a second flowchart illustrating the navigation path planning method provided in this application embodiment. After step S130, the method may further include step S150.

[0084] Step S150: Before the working equipment performs the operation of the next crop row, a second navigation path is determined from the obtained candidate navigation paths based on the driving direction information of the working equipment.

[0085] In this embodiment, optionally, during the operation of a group of crop rows, the working equipment can execute steps S110 to S140 at certain intervals. After completing the operation on the current group of crop rows, it needs to turn to operate on the next group of crop rows. However, navigation path planning requires corresponding images and time. To enable the working equipment to quickly enter the next crop row, a second navigation path can be determined from the obtained candidate navigation paths based on the driving direction information of the working equipment. The second navigation path is used to guide the working equipment to operate on the next crop row. The working equipment can then operate on the next crop row using the second navigation path.

[0086] For example, when obtaining the current first navigation path, the navigation paths corresponding to other crop rows adjacent to the current first navigation path can be obtained as candidate navigation paths, and the actual position of the working equipment corresponding to the candidate navigation paths can be obtained at the same time. After the working equipment completes the journey along the current first navigation path, the second navigation path can be selected from the candidate navigation paths based on the traveling direction of the working equipment (such as turning) and the current position of the working equipment.

[0087] For example, if the first path and the second path that are adjacent to the current first navigation path are obtained, and the working equipment turns left after the current first navigation path is completed, it means that the working equipment needs to enter the crop rows on the left. Therefore, the first path can be selected as the second navigation path to continue driving, and then the equipment can travel along the second navigation path.

[0088] As one possible implementation, after obtaining the first navigation path, the path can be offset towards both sides of the first navigation path according to the path interval distance to obtain candidate navigation paths. The path interval distance represents the spacing between paths that the working equipment can travel on. This path interval distance can be pre-set based on the crop row width and the spacing between adjacent crop rows. For example, assuming there are crop rows A, B, C, and D from left to right, and the working equipment is currently traveling between rows B and C, and may next travel between rows A and B or between rows C and D, the path used when traveling between rows A and B can be offset to the left by one crop row width and one crop row spacing to obtain a candidate navigation path located between rows A and B; similarly, the path used when traveling between rows A and B can be offset to the right by one crop row width and one crop row spacing to obtain a candidate navigation path located between rows C and D.

[0089] As another possible implementation, when the area corresponding to the target bird's-eye view can be used to obtain multiple navigation paths, it can be achieved through... Figure 8 The candidate navigation path is obtained in the manner shown. In this method, step S140 may further include sub-steps S145 to S146.

[0090] Sub-step S145: Based on the center line of the field of view of the target bird's-eye view, determine the second planning range corresponding to the candidate navigation path from the target bird's-eye view.

[0091] Similarly, the coordinates corresponding to the second area to be planned can be preset, and then, when obtaining the target bird's-eye view, the second area to be planned can be determined from the target bird's-eye view based on these coordinates. Alternatively, the relationship between the center line of the field of view of the target bird's-eye view and the second area to be planned can be obtained during field-of-view calibration, and then the second area to be planned in the target bird's-eye view can be determined based on this relationship.

[0092] Sub-step S146: Based on the centerline of the second crop row area included in the second planning range and the preset scale factor, the candidate navigation path is obtained.

[0093] After determining the second planning range, the candidate navigation path can be obtained in the same way as the first navigation path, which will not be elaborated here.

[0094] Optionally, when the operating equipment generally travels between adjacent crop rows, the first navigation path and candidate navigation paths can be obtained in another way. In this method, the center line of the field of view of the target bird's-eye view overlaps with the extension of the centerline of the operating equipment, and the number of crop row regions in the target bird's-eye view is greater than 2. In this case, the center line between any two adjacent crop row regions can be calculated as the second navigation center line. Then, the second navigation center line closest to the center line of the field of view of the target bird's-eye view is taken as the navigation center line corresponding to the first navigation path, and the first navigation path can be obtained by combining it with a preset scale factor. Candidate navigation paths can be obtained based on the remaining second navigation center lines and the preset scale factor.

[0095] To ensure optimal results, path correction may be optionally performed. Please refer to [link / reference]. Figure 9 , Figure 9 This is the third flowchart illustrating the navigation path planning method provided in this application embodiment. In this embodiment, after step S150, the method may further include steps S160 to S190.

[0096] Step S160: During the execution of the second navigation path, a second target image is obtained, and the first target image is updated to the second target image to obtain the target navigation path.

[0097] During the execution of the second navigation path, a captured image can be obtained as the second target captured image. Then, this second target captured image is used as the new first target captured image. After steps S110 to S140, the path to be executed at this time is obtained as the target navigation path. That is, the target navigation path is equivalent to the first navigation path obtained based on the second target captured image.

[0098] Step S170: Determine whether the deviation between the second navigation path and the target navigation path is within a preset range.

[0099] Optionally, the preset range may include a preset angle range and a preset distance range. Lateral deviation and heading deviation can be calculated based on the second navigation path and the target navigation path. These lateral and heading deviations can be calculated based on the current position and attitude of the operating equipment and the target navigation path; or they can be calculated based on the path segments already executed by the operating equipment in the second navigation path and the target navigation path, etc. The lateral and heading deviations can also be calculated in other ways, and no specific calculation method is limited here.

[0100] Optionally, the deviation can be determined to be within the preset range if the lateral deviation is within the preset distance range and the heading deviation is within the preset angle range. Conversely, the deviation can be determined to be outside the preset range if the lateral deviation is within the preset distance range and the heading deviation is within the preset angle range.

[0101] If the deviation is within the preset range, step S180 can be executed: continue executing the second navigation path.

[0102] When the deviation is not within the preset range, step S190 can be executed.

[0103] In step S190, the remaining unexecuted path segments in the second navigation path are corrected according to the target navigation path.

[0104] Optionally, a target path segment corresponding to the remaining unexecuted path segment in the second navigation path can be determined from the target navigation path, and then a path can be planned from the current position of the working equipment to the starting point of the target path segment to execute the target path segment.

[0105] Similarly, during the operation along the target path segment, another target image can be obtained as the new first target image, and the first navigation path and candidate path can be obtained at the same time.

[0106] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a navigation path planning device 200 is given below. Optionally, the navigation path planning device 200 can adopt the above-described... Figure 1 The device structure of the electronic device 100 shown. Further, please refer to... Figure 10 , Figure 10 This is a block diagram of the navigation path planning device 200 provided in this embodiment. It should be noted that the basic principle and technical effects of the navigation path planning device 200 provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The navigation path planning device 200 may include: an interception module 210, a conversion module 220, an identification module 230, and a path acquisition module 240.

[0107] The interception module 210 is used to obtain a partial image of the first target image based on the central axis of the working equipment.

[0108] The conversion module 220 is used to convert the local image to obtain a target bird's-eye view.

[0109] The identification module 230 is used to identify crops in the target bird's-eye view and determine at least one crop row area.

[0110] The path acquisition module 240 is used to obtain a navigation path based on the centerline of the at least one crop row area and a preset scale factor. The preset scale factor represents the relationship between the distances between pixels in the target bird's-eye view and the actual distances.

[0111] Optionally, in this embodiment, the navigation path may further include candidate navigation paths obtained based on the target bird's-eye view, wherein the candidate navigation paths are adjacent to the first navigation path and correspond to different crop rows. The path acquisition module 240 is further configured to: determine a second navigation path from the obtained candidate navigation paths based on the travel direction information of the working equipment before the working equipment performs the operation on the next crop row; the second navigation path is used to guide the working equipment to perform the operation on the next crop row.

[0112] Optionally, in this embodiment, the path acquisition module 240 is further configured to: obtain a second target image during the execution of the second navigation path, and update the first target image to the second target image to obtain a target navigation path; determine whether the deviation between the second navigation path and the target navigation path is within a preset range; if the deviation is within the preset range, continue executing the second navigation path; if the deviation is not within the preset range, correct the remaining unexecuted path segments in the second navigation path according to the target navigation path.

[0113] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown is either stored in or embedded in the operating system (OS) of the electronic device 100, and can be used by... Figure 1 The processor 120 executes the program. Meanwhile, the data and program code required to execute the above modules can be stored in the memory 110.

[0114] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the navigation path planning method.

[0115] In summary, the embodiments of this application provide a navigation path planning method, apparatus, electronic device, and readable storage medium. Upon obtaining a first target image, a target bird's-eye view is obtained from the first target image by view cropping and field-of-view transformation based on the central axis of the working equipment. By identifying crops within the target bird's-eye view, at least one crop row region is obtained. Then, a navigation path is obtained based on the centerline of the at least one crop row region and a preset scale factor, where the preset scale factor represents the relationship between the distances between pixels in the target bird's-eye view and the actual distances. Thus, by cropping the field of view based on the central axis of the working equipment, while ensuring the acquisition of a local image for navigation path planning, interference from other crop rows can be reduced, and the amount of data processing can be reduced, thereby improving the path acquisition speed. Furthermore, by transforming the local image into a bird's-eye view and using the preset scale factor, the actual positioning problem of crops in monocular vision can be effectively solved based on monocular vision images, thereby obtaining an actual navigation path that can be used for crop operations and improving navigation accuracy.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0117] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0118] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] The above description is merely an optional embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A navigation path planning method, characterized in that, include: Based on the center line of the field of view of the target bird's-eye view of the working equipment, the first planning range corresponding to the navigation path is determined from the target bird's-eye view. The size of the first planning range is determined based on the wheel gauge and crop row spacing of the working equipment, or based on the wheel gauge, crop row width and crop row spacing. A total area consisting of the first crop area included in the first planning range is determined, and the edge of the total area is determined by the edge of the first crop row area outside the first planning range; the center line of the total area is determined, and a first navigation path is obtained based on the center line of the total area and a preset scale factor, wherein the first navigation path is used to guide the operating equipment to operate on the current crop row.

2. The method according to claim 1, characterized in that, In the case where the first planning area includes multiple rows of crops; The step of obtaining the first navigation path based on the centerline of the first crop row area included in the first planning range and a preset scale factor includes: If the number of the first crop row areas is even, the two first crop row areas that are closest to or furthest from the center line of the first planning area will be taken as the first target crop row areas. The first navigation path is obtained based on the centerline between the two first target crop row regions and the preset scale factor.

3. The method according to claim 1, characterized in that, When the first planning area includes multiple rows of crops, obtaining the first navigation path based on the centerline of the first crop row area included in the first planning area and a preset scale factor includes: When the number of the first crop row areas is multiple and odd, the first crop row areas on both sides of the first crop row area closest to the center line of the first planning range are taken as the first target crop row areas, wherein the number of crop row areas between each first target crop row area and the first crop row area closest to the center line of the first planning range is equal. The first navigation path is obtained based on the centerline between the two first target crop row regions and the preset scale factor.

4. The method according to claim 2 or claim 3, characterized in that, The step of obtaining the first navigation path based on the centerline between the two first target crop row regions and a preset scale factor includes: The centerline of each of the two first target crop row regions is obtained by fitting and calculating the two first target crop row regions respectively. The center line between the center lines of the crop rows in each of the first target crop row areas is taken as the first navigation center line; The first navigation path is obtained based on the first navigation centerline and the preset scale factor.

5. The method according to claim 1, characterized in that, When the first planning area includes a row of crops, obtaining the first navigation path based on the centerline of the first crop row area included in the first planning area and a preset scale factor includes: The first navigation path is obtained based on at least one of the preset offset distance and the wheel track of the operating equipment, the preset scale factor, and the centerline of the first crop row area, wherein the preset offset distance is a pre-set pixel distance or an actual distance.

6. The method according to claim 5, characterized in that, The first navigation path is obtained based on at least one of a preset offset distance and the wheelbase of the operating equipment, the preset scale factor, and the centerline of the first crop row area, including: If the preset offset distance is a pixel distance, then the center line of the first crop row area is offset according to the preset offset distance to obtain a navigation line; The first navigation path is obtained based on the navigation line and the preset scale factor.

7. The method according to claim 5, characterized in that, The first navigation path is obtained based on at least one of a preset offset distance and the wheelbase of the operating equipment, the preset scale factor, and the centerline of the first crop row area, including: If the preset offset distance is the actual distance, then the initial navigation path is obtained based on the preset scale factor and the centerline of the first crop row area; The initial navigation path is offset according to the preset offset distance to obtain the first navigation path.

8. The method according to claim 5, characterized in that, The first navigation path is obtained based on at least one of a preset offset distance and the wheelbase of the operating equipment, the preset scale factor, and the centerline of the first crop row area, including: Determine the offset distance based on the wheel track of the operating equipment; The actual path is determined based on the preset scale factor and the centerline of the first crop row area; The actual path is offset based on the offset distance to obtain the first navigation path.

9. The method according to claim 5, characterized in that, The first navigation path is obtained based on at least one of a preset offset distance and the wheelbase of the operating equipment, the preset scale factor, and the centerline of the first crop row area, including: Determine the offset distance based on the wheel track of the operating equipment; An initial navigation path is obtained based on the centerline of the first crop row region and the preset scale factor; Based on the offset distance and the preset offset distance, the initial navigation path is offset to obtain the first navigation path.

10. A navigation path planning device, characterized in that, include: The path acquisition module is used to determine the first planning range corresponding to the navigation path from the target bird's-eye view based on the center line of the field of view of the target bird's-eye view of the working equipment. The size of the first planning range is determined based on the wheel track and crop row spacing of the working equipment, or based on the wheel track, crop row width and crop row spacing. A total area consisting of the first crop area included in the first planning range is determined, and the edge of the total area is determined by the edge of the first crop row area outside the first planning range; the center line of the total area is determined, and a first navigation path is obtained based on the center line of the total area and a preset scale factor, wherein the first navigation path is used to guide the operating equipment to operate on the current crop row.

11. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the navigation path planning method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the navigation path planning method as described in any one of claims 1 to 9.

Images