Method, system for positioning truck bed and automatic management method for excavator

Abstract

This invention discloses a truck bed positioning method, system, and excavator automatic management method. The truck bed positioning method includes: acquiring image sensor parameters and acquiring image data to be tested based on the image sensor; performing target recognition on the image data to be tested to obtain truck outline data and truck model data in the image data; determining the actual coordinates of the center point of the truck outline data according to the image sensor parameters; acquiring a truck bed outline template corresponding to the truck model data; spatially aligning and matching the truck bed outline template with the truck outline data; and determining the truck bed positioning result based on the actual coordinates of the center point and the matched truck bed outline template.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology in construction, specifically to a method and system for positioning truck beds and an automatic management method for excavators. Background Technology

[0002] With the rapid development of infrastructure construction in my country, the accurate location and outline information of truck beds are key to achieving efficient operations in scenarios such as logistics loading and unloading, autonomous driving, and cargo delivery. This is especially true for earthwork operations, which face the dual demands of improving efficiency and ensuring safety.

[0003] However, current traditional excavator operations rely on manual operation, which limits efficiency. During autonomous excavation, excavators have limited perception of surrounding trucks and soil, making it difficult to achieve millisecond-level dynamic response to the spatial posture of the bucket. This results in delayed bucket trajectory planning, which not only leads to low construction efficiency but also increases the risk of collisions.

[0004] Therefore, existing technologies suffer from low construction efficiency due to the low positioning efficiency of truck beds during excavator construction. Summary of the Invention

[0005] This invention provides a method, system, and automatic management method for positioning truck beds, aiming to solve the problem of low construction efficiency caused by low truck bed positioning efficiency in the excavator construction process.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A method for positioning a truck bed, comprising: Obtain image sensor parameters and acquire the image data to be tested based on the image sensor; Target recognition is performed on the image data to be tested to obtain truck outline data and truck model data from the image data to be tested. The actual coordinates of the center point of the truck outline data are determined based on the image sensor parameters. Obtain the truck bed outline template corresponding to the truck model data; The truck bed outline template is spatially aligned and matched with the truck outline data, and the positioning result of the truck bed is determined based on the actual coordinates of the center point and the matched truck bed outline template.

[0007] Optionally, the truck model data includes large trucks, medium trucks, and small trucks; the target recognition of the image data to be tested to obtain truck outline data and truck model data in the image data to be tested includes: The truck model data is obtained by classifying the size of the image data to be tested by training a fully trained target YOLO image detection network model; The truck contour data is obtained by performing contour detection on the image data to be tested using a fully trained target YOLO image detection network model.

[0008] Optionally, each truck is labeled with a model identifier; the target recognition of the image data to be tested, to obtain truck outline data and truck model data in the image data to be tested, includes: Perform text recognition on the image data to be tested to obtain the model identifier text in the image data to be tested; Semantic parsing is performed on the model identifier text to obtain the truck model data; Contour detection is performed on the image data to be tested to obtain the truck contour data.

[0009] Optionally, the step of performing target recognition on the image data to be tested to obtain truck outline data in the image data to be tested includes: When the image data to be tested is color image data, and the outline of a truck cannot be identified in the color image data, the point cloud data corresponding to the image data to be tested is obtained, and the point cloud data is converted into depth map data; Edge extraction and contour reconstruction are performed on the depth map data to obtain the truck contour data.

[0010] Optionally, the truck bed profile template includes a variety of preset truck bed profiles, which include one or more of two-dimensional profile data, three-dimensional profile data, point cloud profile data, and parametric profile models.

[0011] Optionally, the image sensor parameters include intrinsic parameters and extrinsic parameters; determining the actual coordinates of the center point of the truck contour data based on the image sensor parameters includes: Obtain the center point pixel coordinates of the truck outline data; Based on the intrinsic parameters, the center point pixel coordinates are converted into three-dimensional coordinates in the coordinate system of the image sensor itself using a perspective projection model. Based on the extrinsic parameters, the three-dimensional coordinates are converted into the actual coordinates of the center point in the world coordinate system.

[0012] Optionally, the step of spatially aligning and matching the truck bed contour template with the truck contour data, and determining the truck bed positioning result based on the actual coordinates of the center point and the matched truck bed contour template, includes: Obtain the center point of the truck bed plane in the truck outline data; Align the center point of the truck bed outline template with the center point of the truck bed plane; Rotate the truck bed contour template, and iteratively match the truck bed contour template and the truck contour data using the iterative nearest point algorithm until the matching error converges to a preset threshold to obtain the optimal spatial transformation matrix. The coordinates of the center point of the template are replaced with the actual coordinates of the center point, and all coordinate points of the truck bed outline template are updated based on the optimal spatial transformation matrix to obtain the positioning result of the truck bed.

[0013] Optionally, obtaining the center point of the truck bed plane of the truck contour data includes: The point cloud data of the image data to be tested is sequentially subjected to pass-through filtering and outlier removal processing to obtain the first point cloud data; Obtain the center point of the truck's cab, and filter the second point cloud data in the first point cloud data whose distance from the center point of the truck's cab is within a preset distance threshold; Filter out the third point cloud data in the second point cloud data whose normal vector value is greater than a preset normal vector threshold; Obtain at least one planar cluster center point of the third point cloud data; Calculate the distance between each planar cluster center point and the center point of the vehicle front, and retain the planar cluster center points whose distances are within a preset distance range; The centroid of all cluster centers within a preset distance range is determined as the center point of the truck bed plane.

[0014] An automatic management method for excavators, comprising: Automatic acquisition of image data to be tested based on image sensors fixedly installed on the excavator; Based on the truck bed positioning method described above, the positioning result of the truck bed is determined according to the image data to be tested; The excavator determines the target unloading location based on the positioning results.

[0015] A positioning system for a truck bed, comprising: The image acquisition module is used to acquire image sensor parameters and acquire image data based on the image sensor. The target recognition module is used to perform target recognition on the image data to be tested, and obtain truck outline data and truck model data in the image data to be tested. A truck center point determination module is used to determine the actual coordinates of the center point of the truck contour data based on the image sensor parameters. The truck bed outline template comparison module is used to obtain the truck bed outline template corresponding to the truck model data; The truck bed positioning module is used to spatially align and match the truck bed outline template with the truck outline data, and determine the positioning result of the truck bed based on the actual coordinates of the center point and the matched truck bed outline template.

[0016] In this embodiment, the truck outline and model data are quickly extracted by target recognition, and the actual coordinates of the center point of the truck outline are directly calculated by combining the image sensor parameters, avoiding the complex on-site measurement of traditional methods. Then, by retrieving the pre-stored truck bed outline template of the corresponding model and performing spatial alignment and matching with the truck outline, the time of rebuilding the truck bed model is saved, the positioning process is simplified, and the efficiency of truck bed positioning is significantly improved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this embodiment, the accompanying drawings used in the description of the embodiment will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an embodiment of the truck bed positioning method provided by the present invention; Figure 2 A schematic flowchart illustrating an embodiment of the present invention for converting a color image into a depth map for recognition; Figure 3 A flowchart illustrating an embodiment of the truck bed position calculation provided by the present invention; Figure 4 A schematic diagram illustrating the results of an embodiment of point cloud data with the bottom plane of the truck bed provided by the present invention; Figure 5 A schematic diagram illustrating the process results of a rigid transformation of template point cloud to truck bed point cloud according to an embodiment of the present invention; Figure 6 A schematic diagram showing the result of an embodiment of ICP point cloud matching provided by the present invention; Figure 7 A schematic diagram of an embodiment of the truck bed positioning system provided by the present invention; Figure 8 This is a schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0019] The technical solutions in this embodiment will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] In the following description, specific embodiments of the invention will be illustrated with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be referred to several times as being performed by a computer, and computer execution as referred to herein includes operations by a computer processing unit representing electronic signals of data in a structured format. This operation transforms the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise alter the operation of the computer in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the invention described above are not intended to be limiting, and those skilled in the art will understand that many of the steps and operations described below can also be implemented in hardware.

[0021] The terms "module" or "unit" as used herein can be considered as software objects executing on the computing system. The different components, modules, engines, and services described herein can be considered as implementation objects on the computing system. The apparatus and methods described herein are preferably implemented in software, but can also be implemented in hardware, both of which are within the scope of this invention.

[0022] This invention provides a method and system for positioning truck beds and an automatic management method for excavators.

[0023] Please see Figure 1 , Figure 1 A flowchart illustrating an embodiment of the truck bed positioning method provided by the present invention includes: S101: Obtain image sensor parameters and acquire the image data to be tested based on the image sensor; It should be noted that image sensors are devices used to collect visual information, specifically including RGB cameras or depth cameras mounted on excavators, as well as multimodal sensing devices such as LiDAR.

[0024] The image data to be tested refers to the original images collected by the image sensor that need to be identified and analyzed. Specifically, it can be color images or depth maps of truck beds in a construction environment, as well as point cloud data.

[0025] Image sensor parameters refer to intrinsic parameters that describe the optical characteristics of the camera, such as focal length and principal point coordinates, as well as extrinsic parameters that describe the relative position and attitude relationship between the camera and the LiDAR.

[0026] S102: Perform target recognition on the image data to be tested to obtain truck outline data and truck model data in the image data to be tested; Target recognition refers to the extraction of features from the image data to be tested, identifying the geometric contours of the truck bed and its vehicle type, providing a visual constraint area for subsequent precise positioning, thereby reducing the amount of data processing and improving the real-time performance of positioning.

[0027] Truck outline data refers to the bounding box of the truck bed obtained by performing image recognition on the image data to be tested.

[0028] Truck model data is used to determine the specific model of the truck. Based on the truck model data, the corresponding truck bed outline template can be retrieved as a matching benchmark to achieve millisecond-level data matching and positioning.

[0029] S103: Determine the actual coordinates of the center point of the truck outline data based on the image sensor parameters; The actual coordinates of the center point refer to the three-dimensional coordinates of the truck bed center point detected in the image data to be tested in the lidar coordinate system. It is an important reference for the initial pose calculation in the point cloud matching process. Through this coordinate, the two-dimensional detection results in the image domain can be mapped to the three-dimensional space, supporting subsequent point cloud registration and pose optimization.

[0030] S104: Obtain the truck bed outline template corresponding to the truck model data; The truck bed outline template is a predefined standard truck bed point cloud model based on different truck models. The corresponding truck bed outline template can be directly retrieved based on the truck model data. The template includes truck bed geometry, key feature points, standardized point cloud distribution features, and can also include other personalized features that are strongly related to the truck bed structure, such as: loading and unloading port position, side panel tilt angle, tailgate opening and closing status indicator points, etc., which can help the system better identify truck data, thereby improving the matching robustness in complex construction scenarios.

[0031] Specifically, the truck bed outline template includes a variety of preset truck bed outlines, which include one or more of the following: two-dimensional outline data, three-dimensional outline data, point cloud outline data, and parametric outline models. It can also be dynamically loaded according to the actual application scenario, and supports online updates and incremental expansion to ensure that the template library always matches the structural characteristics of newly put into use trucks at the construction site.

[0032] S105: Spatial alignment and matching of the truck bed outline template with the truck outline data, and determination of the truck bed positioning result based on the actual coordinates of the center point and the matched truck bed outline template.

[0033] The positioning result of the truck bed refers to the position data of the truck bed, which can be absolute position data or relative position data relative to the excavator, providing accurate spatial position information for the excavator to autonomously excavate earth.

[0034] In this embodiment, the truck outline and model data are quickly extracted by target recognition, and the actual coordinates of the center point of the truck outline are directly calculated by combining the image sensor parameters, avoiding the complex on-site measurement of traditional methods. Then, by retrieving the pre-stored truck bed outline template of the corresponding model and performing spatial alignment and matching with the truck outline, the time of rebuilding the truck bed model is saved, the positioning process is simplified, and the efficiency of truck bed positioning is significantly improved.

[0035] In one specific embodiment, in S102, when the recognition accuracy requirement is not high, trucks are divided into three size categories: large, medium, and small. The truck model data includes large trucks, medium trucks, and small trucks. Target recognition is performed on the image data to be tested to obtain truck contour data and truck model data, including: classifying the image data to be tested by size using a fully trained target YOLO image detection network model to obtain truck model data; and performing contour detection on the image data to be tested using a fully trained target YOLO image detection network model to obtain truck contour data.

[0036] The YOLO target image detection network model is a lightweight, high-real-time convolutional neural network model. It belongs to the real-time target detection network based on deep learning. By processing the input image data to be tested, it can quickly identify truck beds in the construction environment and output key information such as bounding boxes, category confidence scores and vehicle models, which can meet the millisecond-level response requirements of excavator operations.

[0037] Size classification refers to the process of classifying truck models based on the geometric dimensions of the truck bed, such as length, width, and height. By comparing the extracted point cloud dimensions of the truck bed with the standard dimensions of conventional truck models, and combining this with the vehicle model information output by the YOLO network, the specific category of the truck is determined.

[0038] Contour detection focuses on extracting a continuous set of boundary points on the outer edge of the truck bed. Using edge enhancement algorithms and curvature constraint filtering, a closed contour line is accurately fitted. The resulting truck contour data not only includes the projected boundaries of length, width, and height, but also includes key structural semantic points such as the embedded loading and unloading port notch and the tailgate hinge position, ensuring that the template matching error meets the engineering accuracy requirements during subsequent spatial alignment.

[0039] In this embodiment, a fully trained YOLO image detection network model is used to classify truck models by size (large, medium, and small) and detect truck contours. This not only improves the detection efficiency and accuracy of truck model and contour data in the image under test by utilizing the real-time and accurate characteristics of the YOLO model, but also provides a targeted basis for truck bed positioning through accurate model classification and provides direct shape and position data for truck bed positioning through accurate contour detection. The combination of the two improves the reliability and speed of truck bed positioning.

[0040] In another specific embodiment, when high recognition accuracy is required, each truck is independently labeled, that is, each truck is labeled with a model identifier; target recognition is performed on the image data to be tested to obtain truck outline data and truck model data in the image data to be tested, including: performing text recognition on the image data to be tested to obtain the model identifier text in the image data to be tested; performing semantic parsing on the model identifier text to obtain truck model data; and performing outline detection on the image data to be tested to obtain truck outline data.

[0041] A model identifier is a symbol or code used to uniquely identify a specific truck model. It can be a standardized code printed on the side of the truck bed or on the nameplate in the cab, such as "ZL5250GJB" or "DFH3160", or it can be other adaptive text data related to the truck model, without any restrictions.

[0042] The model identifier text is the model identifier in the actual identified image data to be tested. It is one of many model identifiers and is used to correspond to a specific truck model.

[0043] Text recognition refers to the process of extracting model identifier text from the image data to be tested using optical character recognition (OCR) technology. The detected model identifier text is then compared with pre-labeled model identifiers to determine the truck model data.

[0044] In this embodiment, the model identifier text is obtained by performing text recognition on the image data to be tested, and the truck model data is obtained by semantic parsing. This realizes a dual verification mechanism of model identifier and image features, which improves the robustness and confidence of truck model recognition.

[0045] Furthermore, target recognition is performed on the image data to be tested to obtain truck outline data in the image data to be tested, including: when the image data to be tested is color image data and the truck outline cannot be identified in the color image data, the point cloud data corresponding to the image data to be tested is obtained and the point cloud data is converted into depth map data; edge extraction and outline reconstruction are performed on the depth map data to obtain truck outline data.

[0046] Point cloud data is a collection of three-dimensional spatial points collected by LiDAR. Each point contains three-dimensional coordinate information (such as x, y, and z axis coordinates), which can be used to characterize the spatial geometry of an object.

[0047] Depth map data is a two-dimensional image generated by converting point cloud data. Each pixel value represents the distance (depth information) from that point to the sensor. Depth map data is used to extract truck contours when color image recognition fails. Through edge extraction and contour reconstruction algorithms, the geometric contour features of the truck can be captured. It is not affected by lighting conditions and can effectively solve the problem of color image recognition failure in strong light, shadow or low contrast scenes.

[0048] When a color image fails to identify the truck's outline due to factors such as changes in lighting, occlusion, or missing texture, the point cloud is converted into a depth map and the outline is extracted. Through edge extraction and outline reconstruction algorithms, the depth map can more accurately capture the geometric outline features of the truck, reducing the outline loss caused by image noise or background interference. As a supplementary means of color image recognition, it ensures that the system can still acquire target outline data when a single sensor fails.

[0049] In this embodiment, by acquiring the corresponding point cloud data and converting it into depth map data containing three-dimensional structural information, edge extraction and contour reconstruction are performed based on the depth map data. This overcomes the limitations of two-dimensional information in color images, effectively solves the problem that color images cannot identify truck contours, improves the accuracy and robustness of truck contour data acquisition, and provides a reliable foundation for the accurate positioning of the truck bed in the future.

[0050] In one specific embodiment, please refer to Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the present invention for converting a color image into a depth map for recognition. First, color image data is read. Then, color image recognition processing is performed on the color image, specifically identifying whether a truck outline exists in the color image data. If recognition is successful ("Yes"), the process directly proceeds to the "Reading Radar Point Cloud Information" stage. If color image recognition fails ("No"), further "Depth Map Recognition Processing" is performed. If depth map recognition is successful ("Yes"), the process also proceeds to "Reading Radar Point Cloud Information". If depth map recognition also fails ("No"), an "Error Reporting" operation is executed, indicating that the truck outline cannot be recognized. After successfully reading the radar point cloud information, "Truck Pose Calculation" is performed, ultimately "Obtaining the Truck Pose".

[0051] In one specific embodiment, in S103, the image sensor parameters include intrinsic parameters and extrinsic parameters; determining the actual coordinates of the center point of the truck contour data based on the image sensor parameters includes: obtaining the pixel coordinates of the center point of the truck contour data; converting the pixel coordinates of the center point into three-dimensional coordinates in the coordinate system of the image sensor itself based on the intrinsic parameters through a perspective projection model; and converting the three-dimensional coordinates into the actual coordinates of the center point in the world coordinate system based on the extrinsic parameters.

[0052] The center point pixel coordinates refer to the pixel position of the center of the truck bed bounding box in the image on the two-dimensional image plane, usually expressed as (u, v) coordinates.

[0053] Perspective projection model is a mathematical model that describes how three-dimensional spatial points are mapped to a two-dimensional image plane through camera imaging principles. The core of it is to achieve coordinate transformation through camera intrinsic parameters (such as focal length and principal point offset) and extrinsic parameters (rotation matrix and translation vector). By using the perspective projection model, the pixel coordinates of the bounding box identified in the image are projected onto the LiDAR coordinate system, which can establish the spatial correspondence between the two-dimensional image and the three-dimensional point cloud.

[0054] The coordinate system of the image sensor itself is the camera coordinate system. Its origin is located at the optical center of the camera, the z-axis points to the imaging plane along the optical axis, and the x and y axes are parallel to the u and v axes of the image plane, respectively. In this coordinate system, the pixel coordinates are projected back to the three-dimensional coordinates. Then, combined with the rotation and translation relationships in the extrinsic parameters, the result is mapped to a unified world coordinate system, thereby achieving the accurate positioning of the center point of the truck bed in real physical space.

[0055] The actual coordinates of the center point refer to the coordinates of the center point of the truck bed in three-dimensional physical space. They are usually based on the LiDAR coordinate system (x, y, z). The truck bed area is extracted by transforming the pixel coordinates of the bounding box through a perspective projection model and combining it with point cloud data. The results are obtained through cluster analysis and plane fitting calculations. It is the core parameter for locating the truck bed position when the excavator is operating autonomously.

[0056] In this embodiment, by refining the image sensor parameters into intrinsic and extrinsic parameters and performing coordinate transformation step by step, the accuracy and reliability of truck bed positioning can be significantly improved. First, the center point pixel coordinates of the truck outline data are obtained, providing the foundational data for subsequent transformations. Next, using the intrinsic parameters combined with a perspective projection model, the two-dimensional pixel coordinates are accurately converted into three-dimensional coordinates in the image sensor's own coordinate system. This step considers the influence of internal optical characteristics such as lens distortion and focal length on imaging, ensuring accurate mapping from the image plane to the sensor space. Finally, based on the extrinsic parameters, the three-dimensional coordinates in the sensor coordinate system are further converted into the actual center point coordinates in the world coordinate system, effectively eliminating spatial offsets caused by the image sensor's installation position and orientation, allowing the positioning results to directly reflect the true position of the truck bed in the actual working environment. This hierarchical, multi-parameter collaborative coordinate transformation mechanism comprehensively considers various error factors in the imaging process, thereby achieving high-precision calculation of the actual coordinates of the truck bed's center point, providing crucial positioning technology support for automated truck operations and precise loading and unloading applications.

[0057] It should be noted that since the world coordinate system is a relatively stable global coordinate system, once the actual coordinates of the center point are determined, other spatial data related to the actual coordinates of the center point can be modeled and updated in real time. This can better ensure the stability and consistency of the data. In particular, during the continuous operation of the excavator, it can support the dynamic tracking of the position of the truck bed and the optimization of path planning.

[0058] In one specific embodiment, in S105, the truck bed contour template is spatially aligned and matched with the truck contour data, and the positioning result of the truck bed is determined based on the actual coordinates of the center point and the matched truck bed contour template. This includes: obtaining the center point of the truck bed plane of the truck contour data; aligning the template center point of the truck bed contour template with the center point of the truck bed plane; rotating the truck bed contour template, and iteratively matching the truck bed contour template and the truck contour data using an iterative nearest-point algorithm until the matching error converges to a preset threshold to obtain the optimal spatial transformation matrix; replacing the coordinate values ​​of the template center point with the actual coordinates of the center point, and updating all coordinate points of the truck bed contour template based on the optimal spatial transformation matrix to obtain the positioning result of the truck bed.

[0059] The process of obtaining the truck bed center point from the truck contour data includes: sequentially performing pass-through filtering and outlier removal on the point cloud data of the image data to be tested to obtain first point cloud data; obtaining the truck front center point and filtering second point cloud data from the first point cloud data whose distance from the truck front center point is within a preset distance threshold; filtering third point cloud data from the second point cloud data whose normal vector value is greater than a preset normal vector threshold; obtaining at least one planar cluster center point from the third point cloud data; calculating the distance between each planar cluster center point and the truck front center point, and retaining planar cluster center points whose distance is within a preset distance range; and determining the centroid of all cluster center points within the preset distance range as the truck bed center point.

[0060] The center point of the truck bed plane refers to the geometric center calculated from the bottom plane point cloud inside the truck bed using a plane fitting algorithm. It serves as the reference point for positioning the truck bed area. After performing cluster analysis and plane equation fitting on the bottom plane point cloud of the truck bed, the complete truck bed point cloud data is extracted based on this center point, providing a spatial reference for subsequent pose calculation.

[0061] The template center point refers to the origin of the coordinate system of the predefined truck bed template point cloud, which is usually located at the geometric center of the truck bed (such as the center point of the upper surface of the truck bed). When the corresponding template point cloud is retrieved according to the truck model, the template center point is used as the origin of the matching reference and is used for initial alignment with the center point of the actual truck bed point cloud.

[0062] Coincident alignment refers to replacing the coordinates of the template center point with the actual coordinates of the center point, and locking the template center point and the truck bed plane center point together so that they completely coincide in the spatial coordinate system, thereby providing a unique initial spatial reference for subsequent rotation matching.

[0063] Matching error refers to the distance deviation between corresponding points in the template point cloud and the actual truck bed point cloud during the alignment process. It is a core indicator for evaluating matching accuracy. The Iterative Closest Point Algorithm (ICP algorithm) minimizes the matching error by iteratively optimizing the transformation matrix, and finally brings the error to a preset threshold range, ensuring the accuracy of pose calculation.

[0064] The optimal spatial transformation matrix refers to the rigid transformation matrix (including rotation matrix and translation vector) obtained by the ICP algorithm that minimizes the matching error between the template point cloud and the actual point cloud. It is used to describe the precise pose of the truck bed relative to the sensor coordinate system.

[0065] Pass-through filtering is a point cloud preprocessing algorithm that, by setting a numerical range for a certain coordinate axis (such as the X, Y, or Z axis), retains the point cloud within the range and filters out points outside the range, thereby reducing the processing area and improving computational efficiency.

[0066] Outlier removal is a preprocessing step that identifies and removes noisy or outlier points in a point cloud through statistical analysis or radius search. After pass-through filtering, outlier removal is performed on the data to eliminate LiDAR measurement noise and provide a high-quality point cloud for subsequent planar extraction and clustering.

[0067] The center point of the truck's front refers to the geometric center of the feature point cloud at the top of the truck's front, extracted from the preprocessed point cloud. It is determined by calculating the mean value of the point cloud in the Z direction and is used to help determine the direction of the truck bed. The initial yaw angle is calculated by the projection relationship between the center point of the truck bed and the center point of the truck bed.

[0068] A preset distance threshold is a distance determination condition set during point cloud segmentation or clustering to filter point clouds that meet spatial location constraints. For example, when separating the bottom plane of a truck bed, by limiting the distance threshold from the center point of the truck bed, it is ensured that the extracted point cloud belongs to the truck bed region. The preferred preset distance threshold is 2.5 meters.

[0069] The preset normal threshold is a criterion used to filter point clouds with specific normal vector directions. It is usually represented by the numerical range of the Z component of the normal vector. When the preset normal threshold is 0.8, the point cloud of the planar area perpendicular to the ground, such as the bottom of the truck bed, is extracted by setting a threshold where the Z component is greater than 0.8.

[0070] The planar cluster center point refers to the geometric center of each cluster after performing cluster analysis on the extracted planar point cloud. By applying constraints such as area and height to the planar point cloud, the planar cluster at the bottom of the truck bed is selected, and its center point is used as the positioning reference for the truck bed area.

[0071] The preset distance range is a range of coordinate axis values ​​set in the pass-through filter, used to limit the spatial range of point cloud processing. For example, by setting the distance range along the Z-axis, point clouds above the ground and within the height of the truck bed are retained, reducing interference from irrelevant environmental points.

[0072] Cluster centroids are the geometric centers of each cluster after a point cloud has been divided into different clusters using a clustering algorithm (such as Euclidean clustering). They are used to characterize the spatial location of the cluster. In truck bed point cloud processing, cluster centroids are used to distinguish point cloud regions from different components such as the truck body and truck bed.

[0073] The centroid of a cluster center is the average coordinate of all points within a cluster, and it is a specific method for calculating the cluster center. By calculating the centroid, the spatial location of the cluster can be accurately characterized, thereby determining the center of the bottom plane of the truck bed, which serves as the reference for extracting the point cloud from the truck bed.

[0074] In this embodiment, the truck point cloud data is preprocessed by passing through filtering and removing outliers to extract the center point of the truck head. The bottom planar point cloud of the truck bed is then selected by combining a preset distance threshold and a normal threshold, and the centroid of its cluster center point is calculated as the center point of the truck bed plan. The center point of the predefined truck bed contour template is then aligned with the center point of the truck bed plan. The template and the truck contour data are rotated and matched by an iterative nearest-point algorithm until the matching error converges to a preset threshold to obtain the optimal spatial transformation matrix. Finally, all coordinate points of the template are updated based on this matrix to achieve accurate positioning of the truck bed.

[0075] In one specific embodiment, please refer to Figure 3 , Figure 3 The flowchart of an embodiment of the truck bed pose calculation provided by the present invention is as follows: taking "vehicle type + point cloud" as input, it is divided into two main lines: one is "retrieving the corresponding vehicle type point cloud template" as a reference benchmark; the other is to perform "pass-through filtering", "point cloud rigid transformation" and "outlier filtering" on the input point cloud in sequence to purify the data, then "extracting the front of the vehicle" and calculating the center point of the front of the vehicle, "extracting the vehicle body" and calculating its point cloud normal vector, and then obtaining the inner plane of the truck bed through "extracting the plane" and "plane clustering", and then "calculating the center point of the plane as the center point of the truck bed and fitting the plane". Based on the center point of the truck bed and the fitted plane, the system estimates the translation of the truck bed and extracts the truck bed point cloud. Simultaneously, it estimates the rotation angle of the vehicle body. Finally, the truck bed point cloud is transformed and its pose is calculated by IPC point cloud matching with the retrieved vehicle model point cloud template. After completing the pose calculation, the system outputs the six-degree-of-freedom spatial parameters of the truck bed in real time, including the translation components along the X, Y, and Z axes and the rotation angles around the three axes. At the same time, it triggers alarm logic: when the translation deviation is >12cm or the pitch angle deviation is >1.5°, it is determined to be an abnormal loading and a positioning inaccuracy alarm is immediately pushed to the dispatch terminal.

[0076] In one specific embodiment, please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the results of an embodiment of point cloud data with the bottom plane of the truck bed provided by the present invention. The diagram clearly shows the point cloud clusters of the bottom plane of the truck bed after algorithm processing; please refer to... Figure 5 , Figure 5 This is a schematic diagram illustrating the process results of a rigid transformation from a template point cloud to a truck bed point cloud according to an embodiment of the present invention. During the rigid rotation of the template point cloud, the center point of the template and the center point of the truck bed plane are in a locked alignment state, which not only ensures spatial reference consistency but also significantly improves the matching convergence speed and accuracy. Please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram of the results of an embodiment of ICP point cloud matching provided by the present invention. It can be seen from the figure that the template point cloud and the measured truck bed point cloud highly overlap after iterative optimization.

[0077] The present invention also provides an automatic management method for an excavator, comprising: automatically acquiring image data to be measured based on an image sensor fixedly installed on the excavator; determining the positioning result of the truck bed based on the image data to be measured using the truck bed positioning method described above; and determining the target unloading position of the excavator based on the positioning result.

[0078] In this embodiment, by automatically acquiring the image data to be tested by fixing an image sensor on the excavator, information about the working environment can be collected in real time and conveniently, providing basic data support for subsequent processing. Then, the acquired image data to be tested is processed using the truck bed positioning method described above. This positioning method can accurately identify and determine the position of the truck bed in the image, thereby obtaining a reliable truck bed positioning result. Finally, based on this accurate positioning result, the excavator can accurately calculate and determine the target unloading position, realizing automated and precise control of the excavator unloading process. This effectively avoids positioning errors that may be caused by manual operation, improves unloading efficiency and operational safety, reduces reliance on operator experience, and enhances the intelligence level of excavator operation.

[0079] To facilitate better implementation of the truck bed positioning method provided by this invention, this invention also provides an apparatus based on the above-described truck bed positioning method. The meanings of the terms used are the same as in the truck bed positioning method described above, and specific implementation details can be found in the descriptions of the method embodiments.

[0080] Please see Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the truck bed positioning system provided by the present invention, wherein the truck bed positioning system 700 may include: The image acquisition module 701 is used to acquire image sensor parameters and acquire image data based on the image sensor. The target recognition module 702 is used to perform target recognition on the image data to be tested, and to obtain truck outline data and truck model data in the image data to be tested. Truck center point determination module 703 is used to determine the actual coordinates of the center point of the truck contour data based on the image sensor parameters; Truck bed outline template comparison module 704 is used to obtain the truck bed outline template corresponding to the truck model data; The truck bed positioning module 705 is used to spatially align and match the truck bed contour template with the truck contour data, and determine the positioning result of the truck bed based on the actual coordinates of the center point and the matched truck bed contour template.

[0081] The present invention also provides an electronic device, such as... Figure 8 As shown, Figure 8This is a schematic diagram of an embodiment of the electronic device provided by the present invention, specifically: The electronic device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that... Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 801 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor. The application processor mainly handles the operation of the storage medium, user interface, and application programs, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0082] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store application programs required for operating the storage medium and at least one function (such as sound playback function, image playback function, etc.); the data storage area may store data created according to the use of the electronic device. In addition, the memory 802 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0083] The electronic device also includes a power supply 803 that supplies power to various components. Preferably, the power supply 803 can be logically connected to the processor 801 via a power management storage medium, thereby enabling functions such as charging, discharging, and power consumption management through the power management storage medium. The power supply 803 may also include one or more DC or AC power supplies, recharge storage media, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0084] The electronic device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0085] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 runs the application programs stored in the memory 802 to realize various functions, as follows: Obtain image sensor parameters and acquire the image data to be tested based on the image sensor; Target recognition is performed on the image data to be tested to obtain truck outline data and truck model data from the image data to be tested; Determine the actual coordinates of the center point of the truck outline data based on the image sensor parameters; Obtain the truck bed outline template corresponding to the truck model data; The truck bed outline template is spatially aligned and matched with the truck outline data, and the positioning result of the truck bed is determined based on the actual coordinates of the center point and the matched truck bed outline template.

[0086] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0087] Therefore, the present invention provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to execute the steps in any of the truck bed positioning methods provided by the present invention. For example, the computer program, when loaded by a processor, can execute the following steps: Obtain image sensor parameters and acquire the image data to be tested based on the image sensor; Target recognition is performed on the image data to be tested to obtain truck outline data and truck model data from the image data to be tested; Determine the actual coordinates of the center point of the truck outline data based on the image sensor parameters; Obtain the truck bed outline template corresponding to the truck model data; The truck bed outline template is spatially aligned and matched with the truck outline data, and the positioning result of the truck bed is determined based on the actual coordinates of the center point and the matched truck bed outline template.

[0088] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0089] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0090] Since the computer program stored in the computer-readable storage medium can execute the steps in any of the truck bed positioning methods provided by the present invention, the beneficial effects that any of the truck bed positioning methods provided by the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0091] The above provides a detailed description of a truck bed positioning method, system, and excavator automatic management method provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for positioning a truck bed, characterized in that, include: Obtain image sensor parameters and acquire the image data to be tested based on the image sensor; Target recognition is performed on the image data to be tested to obtain truck outline data and truck model data from the image data to be tested. The actual coordinates of the center point of the truck outline data are determined based on the image sensor parameters. Obtain the truck bed outline template corresponding to the truck model data; The truck bed outline template is spatially aligned and matched with the truck outline data, and the positioning result of the truck bed is determined based on the actual coordinates of the center point and the matched truck bed outline template.

2. The truck bed positioning method according to claim 1, characterized in that, The truck model data includes large trucks, medium trucks, and small trucks; the target recognition of the image data to be tested, to obtain truck outline data and truck model data in the image data to be tested, includes: The truck model data is obtained by classifying the size of the image data to be tested by training a fully trained target YOLO image detection network model; The truck contour data is obtained by performing contour detection on the image data to be tested using a fully trained target YOLO image detection network model.

3. The truck bed positioning method according to claim 1, characterized in that, Each truck is marked with a model identifier; the target recognition of the image data to be tested yields truck outline data and truck model data from the image data to be tested, including: Perform text recognition on the image data to be tested to obtain the model identifier text in the image data to be tested; Semantic parsing is performed on the model identifier text to obtain the truck model data; Contour detection is performed on the image data to be tested to obtain the truck contour data.

4. The truck bed positioning method according to claim 1, characterized in that, The process of target recognition in the image data to be tested, to obtain truck outline data in the image data to be tested, includes: When the image data to be tested is color image data, and the outline of a truck cannot be identified in the color image data, the point cloud data corresponding to the image data to be tested is obtained, and the point cloud data is converted into depth map data; Edge extraction and contour reconstruction are performed on the depth map data to obtain the truck contour data.

5. The truck bed positioning method according to claim 1, characterized in that, The truck bed profile template includes a variety of preset truck bed profiles, which include one or more of two-dimensional profile data, three-dimensional profile data, point cloud profile data, and parametric profile models.

6. The truck bed positioning method according to claim 1, characterized in that, The image sensor parameters include intrinsic and extrinsic parameters; determining the actual coordinates of the center point of the truck contour data based on the image sensor parameters includes: Obtain the center point pixel coordinates of the truck outline data; Based on the intrinsic parameters, the center point pixel coordinates are converted into three-dimensional coordinates in the coordinate system of the image sensor itself using a perspective projection model. Based on the extrinsic parameters, the three-dimensional coordinates are converted into the actual coordinates of the center point in the world coordinate system.

7. The truck bed positioning method according to claim 1, characterized in that, The step of spatially aligning and matching the truck bed contour template with the truck contour data, and determining the truck bed positioning result based on the actual coordinates of the center point and the matched truck bed contour template, includes: Obtain the center point of the truck bed plane in the truck outline data; Align the center point of the truck bed outline template with the center point of the truck bed plane; Rotate the truck bed contour template, and iteratively match the truck bed contour template and the truck contour data using the iterative nearest point algorithm until the matching error converges to a preset threshold to obtain the optimal spatial transformation matrix. The coordinates of the center point of the template are replaced with the actual coordinates of the center point, and all coordinate points of the truck bed outline template are updated based on the optimal spatial transformation matrix to obtain the positioning result of the truck bed.

8. The truck bed positioning method according to claim 7, characterized in that, The method of obtaining the truck bed center point of the truck contour data includes: The point cloud data of the image data to be tested is sequentially subjected to pass-through filtering and outlier removal processing to obtain the first point cloud data; Obtain the center point of the truck's cab, and filter the second point cloud data in the first point cloud data whose distance from the center point of the truck's cab is within a preset distance threshold; Filter out the third point cloud data in the second point cloud data whose normal vector value is greater than a preset normal vector threshold; Obtain at least one planar cluster center point of the third point cloud data; Calculate the distance between each planar cluster center point and the center point of the vehicle front, and retain the planar cluster center points whose distances are within a preset distance range; The centroid of all cluster centers within a preset distance range is determined as the center point of the truck bed plane.

9. An automatic management method for excavators, characterized in that, include: Automatic acquisition of image data to be tested based on image sensors fixedly installed on the excavator; The truck bed positioning method according to any one of claims 1-8 determines the positioning result of the truck bed based on the image data to be tested; The excavator determines the target unloading location based on the positioning results.

10. A positioning system for a truck bed, characterized in that, include: The image acquisition module is used to acquire image sensor parameters and acquire image data based on the image sensor. The target recognition module is used to perform target recognition on the image data to be tested, and obtain truck outline data and truck model data in the image data to be tested. A truck center point determination module is used to determine the actual coordinates of the center point of the truck contour data based on the image sensor parameters. The truck bed outline template comparison module is used to obtain the truck bed outline template corresponding to the truck model data; The truck bed positioning module is used to spatially align and match the truck bed outline template with the truck outline data, and determine the positioning result of the truck bed based on the actual coordinates of the center point and the matched truck bed outline template.

Images