Landing leg control method and device for work vehicle, work vehicle, and medium

Abstract

The invention discloses a supporting leg control method and device of an operation vehicle, the operation vehicle and a medium, and relates to the technical field of fire fighting. The method comprises the steps that an image and a depth map, collected by an image collection device, of a target area are obtained, point cloud is obtained according to the depth map, and the target area is an area where supporting legs of an operation vehicle are about to move; according to the point cloud, determining whether there is an obstacle in the target area; determining the ground flatness of the target area according to the point cloud; the ground type of the target area is determined according to the image, and the ground type comprises a non-supportable ground and a supportable ground; determining whether a dangerous target exists in the target area according to the image; and determining a landing leg unfolding strategy according to the existence of the obstacle and / or the ground flatness and / or the ground type and / or the existence of the dangerous target in the target area. According to the method, multiple factors in the area where the supporting leg is about to move are judged, then the supporting leg unfolding strategy is determined based on the multiple factors, and the operation safety of the supporting leg is improved.

Description

Technical Field

[0001] This application relates to the field of fire protection technology, specifically to a method, device, vehicle, and medium for controlling the outriggers of a work vehicle. Background Technology

[0002] During firefighting operations, the lives of firefighters are threatened by factors such as high smoke density and building collapse. Therefore, unmanned fire trucks have begun to replace human firefighters in firefighting missions. An unmanned fire truck is a special vehicle that does not require a human driver and performs firefighting tasks through remote control, autonomous navigation, or a combination of both. It integrates robotics, artificial intelligence, sensor systems, the Internet of Things, and firefighting equipment, aiming to improve firefighting efficiency, reduce firefighter risks, and cope with complex fire scene environments.

[0003] In complex rescue scenarios, unmanned fire trucks need to quickly deploy outriggers to stabilize the vehicle. Currently, outrigger deployment often relies on manual on-site observation. However, since unmanned fire trucks are typically operated remotely, operators have difficulty observing closely, potentially overlooking safety risks and easily leading to accidents. Other operational vehicles, such as aerial ladder trucks, also face similar safety risks.

[0004] It is evident that existing work vehicles pose safety risks when their outriggers are deployed. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, vehicle, and medium for controlling the outriggers of a work vehicle, in order to solve the technical problem of safety risks in work vehicles when the outriggers are deployed in the prior art.

[0006] To achieve the above objectives, the first aspect of this application provides a method for controlling the outriggers of a work vehicle, comprising: The image and depth map of the target area are acquired by the image acquisition device, and the point cloud is obtained based on the depth map. The target area is the area where the outriggers of the work vehicle are to move. Based on the point cloud, determine whether there are obstacles in the target area; Determine the ground flatness of the target area based on the point cloud; Based on the image, determine the ground type of the target area, which includes unsupportable ground and supportable ground; Based on the image, determine whether there are any dangerous targets in the target area; Based on the presence of obstacles and / or ground flatness and / or ground type and / or the presence of hazardous targets in the target area, a deployment strategy for the outriggers is determined. The outrigger deployment strategy includes allowing deployment, recommending deployment after adjustments, and prohibiting deployment.

[0007] In this embodiment of the application, determining whether there are obstacles in the target area based on the point cloud includes: Determine the ground elevation of each point in the point cloud based on the installation location of the image acquisition device; Based on the height of each point above the ground, all points are divided into ground point cloud and non-ground point cloud; Cluster the non-terrestrial point clouds to obtain at least one point cloud cluster; If the number of points in a point cloud cluster exceeds the preset number of points, an obstacle is identified. If the number of points in a point cloud cluster does not exceed the preset number of points, it is determined that there are no obstacles.

[0008] In this embodiment of the application, determining the ground flatness of the target area based on the point cloud includes: Determine the covariance matrix of the ground point cloud; The eigenvector corresponding to the smallest eigenvalue in the covariance matrix is ​​used as the plane normal vector, and the equation representing the ground plane is determined based on the plane normal vector. Based on each point in the ground point cloud and the equation representing the ground plane, determine the distance between each point in the ground point cloud and the ground plane; Based on distance, determine the category of each point in the ground point cloud, where the categories include ground protrusions and ground depressions; If the combined area of ​​points in the same category of ground point cloud exceeds a preset area threshold, the ground flatness of the target area is determined to be uneven.

[0009] In this embodiment of the application, determining the category of each point in the ground point cloud based on distance includes: If the distance is greater than the first preset distance threshold and the point is located above the plane, the point is determined as a ground protrusion point; If the distance is greater than the second preset distance threshold and the point is located below the plane, the point is determined as the ground collapse point.

[0010] In this embodiment of the application, determining the ground type of the target area based on the image includes: The target image is obtained by cropping the region of interest (ROI) of the image. The target image is input into a preset segmentation model to obtain the ground type of the target area.

[0011] In this embodiment of the application, determining whether a dangerous target exists in the target area based on the image includes: The image is input into a preset image detection model to obtain the image detection result; Map the image detection results to the depth map coordinate system; If the image detection result is within the preset radius of the outrigger, a dangerous target is determined to exist in the target area.

[0012] In the embodiments of this application, If there are no obstacles in the target area, the ground is flat, the ground type is supportable, and there are no dangerous targets, the outrigger deployment strategy is determined to be deployment permitted. In the case of obstacles in the target area, determine whether there is a target safety zone between the obstacle and the fully retracted outrigger position. The target safety zone meets the following conditions: the ground is flat, the ground type is a supportable ground, there are no dangerous targets, and the size is greater than a preset size threshold. If there are no obstacles in the target area, determine whether there is a target safe area between the outrigger's maximum span position and the outrigger's fully retracted position; In the presence of a target safe zone, the recommended deployment strategy for the outriggers is to deploy them after adjustments. In the absence of a target safe zone, the outrigger deployment strategy is determined to be prohibit deployment.

[0013] A second aspect of this application provides a leg control device for a work vehicle, comprising: The memory is configured to store instructions; and The processor is configured to retrieve instructions from memory and, when executing the instructions, to implement the outrigger control method of the work vehicle according to the first aspect.

[0014] A third aspect of this application provides a work vehicle, comprising: Support legs; According to the outrigger control device of the second aspect of the work vehicle.

[0015] A fourth aspect of this application provides a machine-readable storage medium storing instructions for causing a machine to perform a leg control method for a work vehicle according to the first aspect.

[0016] By using the above technical solution, images and point clouds acquired by the image acquisition device are used to determine multiple factors in the area where the outriggers are to be moved. Then, based on the comprehensive determination of multiple factors, the outrigger deployment strategy is determined to assist the operator in judging whether the outriggers are suitable for deployment, thereby improving the safety of outrigger operations.

[0017] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 The schematic diagram illustrates a flowchart of a method for controlling the outriggers of a work vehicle according to an embodiment of this application; Figure 2 The schematic diagram illustrates the structure of a leg control device for a work vehicle according to an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0020] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0021] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0022] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0023] Figure 1The illustration schematically shows a flowchart of a method for controlling the outriggers of a work vehicle according to an embodiment of this application. Figure 1 As shown in the figure, this application embodiment provides a method for controlling the outriggers of a work vehicle, which may include the following steps 110-160.

[0024] Step 110: Acquire the image and depth map of the target area captured by the image acquisition device, and obtain the point cloud based on the depth map. The target area is the area where the outriggers of the work vehicle are to move.

[0025] In this embodiment, the image acquisition device can be a depth camera or a depth camera combined with a lidar. A depth camera is a device capable of acquiring depth information of objects in a scene (i.e., the distance from the object to the camera). Unlike traditional cameras that can only capture two-dimensional planar images, depth cameras can not only record the color and texture of objects, but also measure and output the three-dimensional spatial coordinates of objects through specific techniques, thereby constructing a three-dimensional model of the scene. The data output by the depth camera is usually presented in the form of a depth map. The value of each pixel in the depth map represents the distance from the corresponding object to the camera; the larger the value, the farther the distance. A lidar (Light Detection and Ranging) is a radar system that uses emitted laser beams to detect the position, velocity, and other characteristics of a target. The working principle of lidar is to emit a detection signal (laser beam) towards the target, then compare the received signal reflected back from the target (target echo) with the emitted detection signal. After appropriate processing, information about the target can be obtained, such as target distance, azimuth, height, velocity, attitude, and even shape parameters, thereby enabling target detection, tracking, and identification.

[0026] It is understood that the embodiments of this application do not limit the installation position of the image acquisition device. In a preferred embodiment, the image acquisition device is mounted on a support leg to more accurately acquire images and depth maps of the target area.

[0027] Based on the obtained depth map, it can be transformed to obtain the point cloud of the target area.

[0028] Step 120: Based on the point cloud, determine whether there are obstacles in the target area.

[0029] In this embodiment, since point clouds can be used to describe the spatial distribution and surface characteristics of objects, it is possible to determine whether obstacles exist in the target area, i.e., objects that may affect the movement of the outriggers, based on the point cloud. It is understood that when obstacles exist in the target area, the operator needs to be alerted.

[0030] In one optional implementation, step 120 includes: Determine the ground elevation of each point in the point cloud based on the installation location of the image acquisition device; Based on the height of each point above the ground, all points are divided into ground point cloud and non-ground point cloud; Cluster the non-terrestrial point clouds to obtain at least one point cloud cluster; If the number of points in a point cloud cluster exceeds the preset number of points, an obstacle is identified. If the number of points in a point cloud cluster does not exceed the preset number of points, it is determined that there are no obstacles.

[0031] Specifically, the point cloud is first classified into different types. In this embodiment, the classification can be based on height. The height of the point cloud above the ground is determined according to the installation position of the image acquisition device, as shown in the following formula (1): h(point)=Hy*cosβ-z*sinβ (1) In the formula, h(point) represents the height of each point in the point cloud above the ground, H represents the distance between the installation position of the image acquisition device and the ground, y represents the vertical coordinate of each point in the point cloud in the camera coordinate system, β represents the pitch angle of the image acquisition device, and z is the depth coordinate of each point in the point cloud in the camera coordinate system.

[0032] The calculated ground clearance of each point is compared with the ground clearance of the outriggers, where the outrigger's footplate is the height above the ground. If the point's ground clearance is less than the outrigger's ground clearance, the corresponding point is determined to be part of the ground point cloud; if the point's ground clearance is greater than or equal to the outrigger's ground clearance, the corresponding point is determined to be part of the non-ground point cloud. Subsequent steps require different logical processing based on the ground point cloud and the non-ground point cloud.

[0033] For non-terrestrial point clouds, point cloud clustering algorithms are used for clustering. Point cloud clustering algorithms are used to segment point cloud datasets into different categories, effectively detecting and identifying objects in the spatial environment, thus providing a foundation for more advanced computer vision applications. The purpose of point cloud clustering is to divide the point cloud dataset into different categories, each containing points with similar features (such as distance, density, normal vectors, etc.). Illustratively, point cloud clustering algorithms can include DBSCAN (Density-Based Spatial Clustering of Applications with Noise), K-means clustering, and OPTICS (Ordering Points To Identify the Clustering Structure).

[0034] Understandably, if the number of points in a point cloud cluster exceeds the preset number of points, it indicates that the object represented by the point cloud cluster may be large and affect the movement of the outriggers; therefore, it is identified as an obstacle. If the number of points in a point cloud cluster does not exceed the preset number of points, it will not be identified as an obstacle.

[0035] It should be noted that obstacles can be marked in red on the human-machine interface of the monitor to alert the operator.

[0036] In this embodiment, points in the point cloud are divided by height, non-ground point clouds are clustered, and obstacles are determined based on the clustering results, which improves the accuracy of obstacle identification and lays the foundation for subsequent leg deployment decisions.

[0037] Step 130: Determine the ground flatness of the target area based on the point cloud.

[0038] In this embodiment, the ground flatness of the target area can also be determined based on point clouds. Uneven ground, with bumps or depressions, can affect the vertical support of the outriggers. Understandably, operators should be alerted when the ground flatness of the target area is uneven.

[0039] In one optional implementation, step 130 includes: Determine the covariance matrix of the ground point cloud; The eigenvector corresponding to the smallest eigenvalue in the covariance matrix is ​​used as the plane normal vector, and the equation representing the ground is determined based on the plane normal vector. Based on each point in the ground point cloud and the equation representing the ground plane, determine the distance between each point in the ground point cloud and the ground plane; Based on distance, determine the category of each point in the ground point cloud, where the categories include ground protrusions and ground depressions; If the combined area of ​​points in the same category of ground point cloud exceeds a preset area threshold, the ground flatness of the target area is determined to be uneven.

[0040] Specifically, for ground point clouds, the ground representation equation can be constructed based on the eigenvector corresponding to the smallest eigenvalue in the covariance matrix. The ground representation equation can be expressed as the following formula (2): Ax + By + Cz + D = 0 (2) In the formula, (A,B,C) represents the plane normal vector, and D is a constant that can be obtained by the mean of the coordinates of the ground point cloud.

[0041] Based on each point in the ground point cloud and the equation representing the ground plane, the distance between each point in the ground point cloud and the ground plane can be determined by referring to the following formula (3): Δz=|Ax_i+By_i +Cz_i+D| / sqrt{A 2 +B 2 +C 2}(3) In the formula, (Ax_i, By_i, Cz_i) represents the coordinates of each point in the ground point cloud, (A,B,C) represents the plane normal vector, D is a constant, and sqrt represents the square root.

[0042] In this embodiment, each point in the ground point cloud is further classified based on distance, with categories including ground protrusions and ground depressions. It is understood that different types of points can be processed differently in the human-machine interface of the display; for example, ground protrusions are marked in yellow, and ground depressions are marked in purple.

[0043] If the combined area of ​​points of the same category exceeds the preset area threshold, it indicates that there are large areas of ground protrusions or depressions. In this case, the ground flatness of the target area is determined to be uneven, and it may be necessary to remind the operator to pay attention.

[0044] In this embodiment, by constructing a representation equation for the ground plane, the ground point cloud is classified, and the ground flatness is further determined, laying the foundation for the subsequent decision on the deployment of the outriggers.

[0045] In one alternative implementation, determining the category of each point in the ground point cloud based on distance includes: If the distance is greater than the first preset distance threshold and the point is located above the plane, the point is determined as a ground protrusion point; If the distance is greater than the second preset distance threshold and the point is located below the plane, the point is determined as the ground collapse point.

[0046] In this embodiment, if a point in the ground point cloud is more than a first preset distance threshold above the ground plane and is located above the plane, it indicates that the point itself has a certain height and may affect the vertical support of the outrigger. If a point in the ground point cloud is more than a second preset distance threshold below the ground plane and is located below the plane, it indicates that the point itself has a certain depth and may also affect the vertical support of the outrigger. Therefore, it is necessary to identify these points.

[0047] Step 140: Based on the image, determine the ground type of the target area, where the ground type includes unsupportable ground and supportable ground.

[0048] Specifically, based on the images acquired by the image acquisition device, the ground type around the outriggers in the target area can be determined. In this embodiment, the ground type includes unsupportable ground and supportable ground. It is understood that unsupportable ground refers to ground where the outriggers cannot provide support; if the outriggers move onto such ground, it will have a certain impact on the safe operation of the work vehicle, such as grass or mud. Supportable ground refers to ground where the outriggers can provide support; generally, it will not affect the safe operation of the work vehicle, such as concrete or asphalt. It is understood that when the ground type is unsupportable, the operator needs to be alerted.

[0049] In one optional implementation, step 140 includes: The target image is obtained by cropping the region of interest (ROI) of the image. The target image is input into a preset segmentation model to obtain the ground type of the target area.

[0050] In this embodiment, the image is first cropped to extract the ROI (Region of Interest). Then, a preset segmentation model is used to obtain the ground type of the target region.

[0051] The preset segmentation model can be a trained DeepLabv3+ model. During model training, multiple images of various ground types, including cement, asphalt, flagstone, compacted soil, sand, grassland (at different heights and with varying moisture levels), mud, gravel, and ice, can be collected under different seasons and weather conditions (sunny, rainy, foggy, and post-snow). These images are then fused with existing urban landscape and topographical semantic segmentation datasets (such as Cityscapes and some subclasses of ADE20K) to supplement specific ground type samples, resulting in a sample set. This sample set is then divided into training, validation, and test sets in an 8:1:1 ratio. Using professional image annotation tools (such as LabelMe), precise pixel-level polygon or mask annotations are performed on each ground type (cement, asphalt, grassland, mud, sand, gravel, and unknown / other) to ensure clear and accurate annotation boundaries. Furthermore, data augmentation techniques such as randomly adjusting brightness, contrast, saturation, and Gamma values, simulating different time periods and ambient light, adding Gaussian noise, simulating rain and fog, and random rotation can be used to process the sample set in order to improve the generalization of the trained preset segmentation model.

[0052] In this embodiment of the application, a deep learning model is used to segment the ground of the target area, laying the foundation for determining the subsequent deployment decision of the outriggers.

[0053] Step 150: Based on the image, determine whether there are dangerous targets in the target area.

[0054] Specifically, based on the images acquired by the image acquisition device, dangerous targets in the target area can be identified. In this embodiment, dangerous targets include, but are not limited to, manhole covers, trench covers, and other targets that may pose a certain danger to the support of the outriggers. It is understood that when a dangerous target is detected, the operator needs to be alerted.

[0055] In one optional implementation, step 150 includes: The image is input into a preset image detection model to obtain the image detection result; Map the image detection results to the depth map coordinate system; If the image detection result is within the preset radius of the outrigger, a dangerous target is determined to exist in the target area.

[0056] In this embodiment, a preset image detection model is used to obtain the image detection result. Since the image detection result is planar, it needs to be mapped to a depth map coordinate system to correspond to the object in the depth map. Therefore, based on the positional relationship between the image detection result and the outrigger, it can be determined whether the image detection result will affect the movement of the outrigger, thus identifying whether the image detection result is a dangerous target. It should be noted that the outrigger can only move within a certain range; therefore, the preset radius range of the outrigger can be understood as the outrigger's movement area.

[0057] The preset image detection model can be a trained YOLO (You Only Look Once) model. During model training, multiple images of dangerous targets, including manhole covers, ditch covers, and balls, can be collected, covering urban roads, construction sites, wilderness, rainy, snowy, foggy, and hazy weather, nighttime (with or without supplemental lighting), and scenes with partial target occlusion, to obtain a sample set. This sample set is then divided into training, validation, and test sets in an 8:1:1 ratio. Rotated bounding boxes are used, with the labeled boundaries closely adhering to the target edges. Furthermore, geometric transformations (rotation, scaling 0.5-1.5x, cropping, etc.), photometric distortion (brightness / contrast jitter, color channel shift), and data augmentation techniques such as adding Gaussian noise, simulating rainy / foggy weather, and random rotation can be used to process the sample set, thereby improving the generalization ability of the trained preset image detection model.

[0058] In this embodiment of the application, a deep learning model is used to identify dangerous targets, laying the foundation for determining the subsequent deployment decision of the outriggers.

[0059] Step 160: Determine the outrigger deployment strategy based on whether there are obstacles and / or ground flatness and / or ground type and / or dangerous targets in the target area, wherein the outrigger deployment strategy includes allowing deployment, recommending deployment after adjustment, and prohibiting deployment.

[0060] In this embodiment of the application, based on the four results obtained above, a comprehensive outrigger deployment strategy is determined to assist the operator, thereby avoiding accidents caused by collisions or ground collapses when the outriggers are deployed, and improving the safety of the operation of the work vehicle.

[0061] The following is an illustrative description of the specific outrigger deployment strategy. It is understood that the outrigger deployment strategy can be adjusted according to actual needs, and the embodiments of this application do not limit it.

[0062] In one optional implementation, step 160 includes: If there are no obstacles in the target area, the ground is flat, the ground type is supportable, and there are no dangerous targets, the outrigger deployment strategy is determined to be deployment permitted. In the case of obstacles in the target area, determine whether there is a target safety zone between the obstacle and the fully retracted outrigger position. The target safety zone meets the following conditions: the ground is flat, the ground type is a supportable ground, there are no dangerous targets, and the size is greater than a preset size threshold. If there are no obstacles in the target area, determine whether there is a target safe area between the outrigger's maximum span position and the outrigger's fully retracted position; In the presence of a target safe zone, the recommended deployment strategy for the outriggers is to deploy them after adjustments. In the absence of a target safe zone, the outrigger deployment strategy is determined to be prohibit deployment.

[0063] Specifically, if the target area has no obstacles, the ground is flat, the ground type is supportable, and there are no dangerous targets, it means that there are no factors in the target area that may pose a danger to the deployment of the outriggers. Therefore, the outriggers can be deployed, and the deployment strategy is determined to be deployment permitted.

[0064] If obstacles exist in the target area, a safe zone can be found between the obstacle and the fully retracted outrigger position. Similarly, if no obstacles exist, a safe zone can be found between the maximum outrigger span and the fully retracted outrigger position. A safe zone must meet four conditions: the ground must be flat; the ground type must be supportable; there must be no dangerous targets; and the zone's size must be greater than a preset size threshold. The preset size threshold is generally the size of the outrigger pad, meaning it must meet the size requirement for the outrigger to be lowered. If a safe zone exists, the outrigger can be adjusted and deployed within it, and this zone can be marked in green on the human-machine interface. If no safe zone exists, there is no space for the outrigger to deploy; therefore, the outrigger deployment strategy is set to prohibit deployment, and an alarm can be triggered, indicating the specific risk type.

[0065] This application embodiment uses images and point clouds acquired by an image acquisition device to determine multiple factors in the area where the outriggers are to be moved. Based on these multiple factors, a comprehensive outrigger deployment strategy is determined to assist operators in judging whether outrigger deployment is appropriate, thereby improving the safety of outrigger operations.

[0066] Figure 2 This schematically illustrates a structural block diagram of an outrigger control device for a work vehicle according to an embodiment of this application. Figure 2 As shown in the figure, this application embodiment provides a leg control device for a work vehicle, which may include: Memory 210 is configured to store instructions; and The processor 220 is configured to retrieve instructions from the memory 210 and, when executing the instructions, to implement the aforementioned method for controlling the boom.

[0067] The outrigger control device for the work vehicle provided in this application embodiment can realize each process of the outrigger control method for the work vehicle in the method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0068] This application embodiment also provides a work vehicle, which may include: Support legs; The outrigger control device of the above-mentioned work vehicle.

[0069] This application also provides a machine-readable storage medium storing instructions for causing a machine to perform the aforementioned outrigger control method for a work vehicle.

[0070] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0075] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0076] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0077] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0078] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method of controlling a support leg of a work vehicle, characterized by, The method comprises the following steps: acquiring an image and a depth map of a target region collected by an image collection device, and obtaining a point cloud according to the depth map, wherein the target region is a region where a support leg of a work vehicle is to be moved; determining whether there is an obstacle in the target region according to the point cloud; determining the ground flatness of the target region according to the point cloud; determining the ground type of the target region according to the image, wherein the ground type includes non-supportable ground and supportable ground; determining whether there is a dangerous target in the target region according to the image; determining a support leg deployment strategy according to whether there is an obstacle in the target region and / or the ground flatness and / or the ground type and / or whether there is a dangerous target, wherein the support leg deployment strategy includes allowing deployment, suggesting adjusted deployment, and prohibiting deployment.

2. The method of claim 1, wherein, The method of determining whether there is an obstacle in the target region according to the point cloud comprises: determining the ground clearance of each point in the point cloud according to the installation position of the image collection device; dividing all points into ground point clouds and non-ground point clouds according to the ground clearance of each point; clustering the non-ground point clouds to obtain at least one point cloud cluster; determining that there is an obstacle if the number of point clouds in the point cloud cluster exceeds a preset number of point clouds; determining that there is no obstacle if the number of point clouds in the point cloud cluster does not exceed the preset number of point clouds.

3. The method of claim 2, wherein, The method of determining the ground flatness of the target region according to the point cloud comprises: determining the covariance matrix of the ground point clouds; determining a plane normal vector as an eigenvector corresponding to the smallest eigenvalue in the covariance matrix, and determining a representation equation of a ground plane according to the plane normal vector; determining the distance between each point in the ground point clouds and the ground plane according to each point in the ground point clouds and the representation equation of the ground plane; determining the category of each point in the ground point clouds according to the distance, wherein the category includes ground protruding points and ground recessed points; determining that the ground flatness of the target region is uneven if the enclosed area of points in the ground point clouds of the same category exceeds a preset area threshold.

4. The method of claim 3, wherein, The method of determining the category of each point in the ground point clouds according to the distance comprises: determining that the point is a ground protruding point if the distance is greater than a first preset distance threshold and is located above the plane; determining that the point is a ground recessed point if the distance is greater than a second preset distance threshold and is located below the plane.

5. The method of claim 1, wherein, The method of determining the ground type of the target region according to the image comprises: performing ROI cropping on the image to obtain a target image; inputting the target image into a preset segmentation model to obtain the ground type of the target region.

6. The method of claim 1, wherein, The method of determining whether there is a dangerous target in the target region according to the image comprises: inputting the image into a preset image detection model to obtain an image detection result; mapping the image detection result to a depth map coordinate system; determining that there is a dangerous target in the target region if the image detection result is located within a preset radius range of the support leg.

7. The method of claim 1, wherein, determining the landing leg deployment strategy according to whether there is an obstacle in the target area and / or the ground flatness and / or the ground type and / or whether there is a dangerous target, comprising: in the case that there is no obstacle in the target area, the ground flatness is flat, the ground type is supportable ground, and there is no dangerous target, determining the landing leg deployment strategy as being allowed to deploy; in the case that there is an obstacle in the target area, determining whether there is a target safety area between the obstacle and the landing leg full retracted position, wherein the target safety area satisfies the following conditions: the ground flatness is flat, the ground type is supportable ground, there is no dangerous target, and the size is greater than a preset size threshold; in the case that there is no obstacle in the target area, determining whether there is a target safety area between the landing leg maximum span position and the landing leg full retracted position; in the case that there is a target safety area, determining the landing leg deployment strategy as being suggested to adjust and deploy; in the case that there is no target safety area, determining the landing leg deployment strategy as being prohibited to deploy.

8. A control device for a support leg of a work vehicle, characterized by comprising: a memory configured to store instructions; and a processor configured to call the instructions from the memory and enable the landing leg control method of the work vehicle according to any one of claims 1 to 7 to be implemented when the instructions are executed.

9. A work vehicle characterized by, comprising: a landing leg; the landing leg control device of the work vehicle according to claim 8.

10. A machine-readable storage medium, characterized in that, The machine readable storage medium has instructions stored thereon for causing a machine to perform the landing leg control method of the work vehicle according to any one of claims 1 to 7.

Images