An obstacle detection method and device for unmanned motor car

Abstract

This application relates to an obstacle detection method and apparatus for unmanned electric locomotives. The locomotive has a camera and a lidar mounted on its front end. The obstacle detection method for unmanned electric locomotives includes: determining an image plane and a scene image; projecting point cloud data from the lidar onto the image plane; determining track point cloud data within the track area; constructing a passage space based on the track point cloud data and the locomotive's three-dimensional dimensions; clustering the point cloud data; if the three-dimensional space of a point cloud cluster overlaps with the passage space, the point cloud cluster is considered an obstacle to be detected; for any obstacle to be detected, acquiring the point cloud trajectory of the obstacle and its running trajectory in the scene image; determining the target area of ​​the point cloud trajectory and the two-dimensional area of ​​the running trajectory in the image plane; when the overlap rate between the two-dimensional area and the target area is greater than a preset threshold, outputting the obstacle type and location information. This application improves the accuracy of obstacle detection.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to an obstacle detection method and apparatus for unmanned electric locomotives. Background Technology

[0002] Electric locomotives are specialized industrial vehicles powered by electricity that operate in enclosed environments such as mines and tunnels. In existing technologies, a forward-facing scene image is obtained using an onboard camera at the front of the locomotive. This image is then processed using traditional visual methods or deep learning to detect the track area, determine the locomotive's travel path, and identify obstacles based on this path. The system then determines whether the obstacles are located within the track area and whether they would encroach on the locomotive's passageway, thus performing obstacle detection.

[0003] However, judging based on the forward scene image of the vehicle-mounted camera is essentially based on a two-dimensional image space, which makes it difficult to obtain the three-dimensional relationship between the obstacle and the locomotive. Furthermore, the forward scene image obtained by the vehicle-mounted camera is easily affected by environmental factors such as changes in lighting, dust, and low light, resulting in blurry images and unclear image textures. This makes it difficult to accurately detect obstacles when using images, leading to low accuracy in obstacle detection results. Summary of the Invention

[0004] This application provides an obstacle detection method and apparatus for unmanned electric locomotives, which at least solves the problem of low obstacle detection accuracy during the operation of unmanned electric locomotives in related technologies.

[0005] In a first aspect, embodiments of this application provide an obstacle detection method for an unmanned electric locomotive, wherein a camera and a lidar are installed at the front end of the locomotive body, and the method includes: Based on the camera device, an image plane and a scene image located on the image plane are determined. A preset detection algorithm is used to identify the track region in the scene image. The point cloud data of the lidar is projected onto the image plane to determine the track point cloud data located within the track region. Based on the track point cloud data and the three-dimensional dimensions of the electric locomotive, the passage space of the electric locomotive is constructed. Clustering the point cloud data that meets the preset height yields several point cloud clusters. If the three-dimensional space of the point cloud cluster overlaps with the passage space, the point cloud cluster is used as an obstacle to be tested. For any of the obstacles to be tested, the point cloud trajectory of the obstacle to be tested and the running trajectory of the scene image in continuous frame mode are acquired within a preset time period. The point cloud trajectory is projected onto the image plane, and the target region of the point cloud trajectory and the two-dimensional region of the running trajectory are determined in the image plane. When the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, the obstacle type of the scene image and the position information of the point cloud cluster are output.

[0006] In one embodiment, constructing the passage space for the electric locomotive based on the track point cloud data and the three-dimensional dimensions of the locomotive includes: A three-dimensional coordinate system is constructed with the front of the electric locomotive as the origin of the three-dimensional coordinate system, the forward direction of the electric locomotive as the positive x-axis, the rightward direction perpendicular to the x-axis as the positive y-axis, and the upward direction perpendicular to the xy plane as the positive z-axis. The spatial boundaries are determined based on the three-dimensional coordinate system and the three-dimensional dimensions. The direction of the x-axis is updated based on the orbit point cloud data, and the orbit is divided into different segments along the x-axis at preset intervals. For any given segment, the center coordinates are determined based on all the orbital point cloud data within the segment, and the passage space for the segment is constructed based on the center coordinates and the spatial boundaries.

[0007] In one embodiment, the clustering of point cloud data satisfying a preset height yields several point cloud clusters. If the three-dimensional space of the point cloud clusters overlaps with the passage space, the point cloud clusters are used as obstacles to be measured, including: Based on the orbit point cloud data, a orbit plane is fitted. In the vertical direction of the orbit plane, the point cloud data that meets the preset height is filtered to obtain intermediate point cloud data. The orbit plane is located in the xy plane of the three-dimensional coordinate system. Clustering the intermediate point cloud data yields several point cloud clusters, determining the three-dimensional space of each point cloud cluster, and based on the segment of the three-dimensional space located on the x-axis, determining the passage space where the point cloud cluster is located; If the three-dimensional space of the point cloud cluster overlaps with the passage space in both the y-axis and z-axis directions, then the point cloud cluster is considered as the obstacle to be measured.

[0008] In one embodiment, acquiring the point cloud trajectory of the obstacle to be measured within a preset time period includes: Based on the point cloud clusters within the passage space, the geometric features of the point cloud clusters are obtained, and the geometric features include the centroid; Within the preset time period, the centroid is updated temporally using a preset filtering algorithm to obtain the point cloud trajectory.

[0009] In one embodiment, the geometric features include the bounding box of the point cloud cluster. The step of projecting the point cloud trajectory onto the image plane, determining the target region of the point cloud trajectory and the two-dimensional region of the trajectory in the image plane, and outputting the obstacle type and the position information of the point cloud cluster in the scene image when the overlap rate between the two-dimensional region and the target region is greater than a preset threshold includes: Target detection is performed on the scene images in consecutive frames to determine the obstacle type of the obstacle, and the position information of the obstacle is determined based on the geometric features of each point cloud cluster in the point cloud trajectory. The point cloud trajectory is projected onto the image plane, and the target region of each point cloud cluster in the point cloud trajectory is determined according to the bounding box in the image plane. The two-dimensional region of the obstacle to be tested in each frame of the scene image is determined by the target detection algorithm. Obtain the intersection and union of the target region and the two-dimensional region, determine the overlap rate based on the ratio of the intersection to the union, and output the obstacle type and the location information of the point cloud cluster when the overlap rate is greater than the preset threshold.

[0010] In one embodiment, after outputting the obstacle type and point cloud cluster location information of the scene image when the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, the method further includes: When the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, a fused trajectory is generated based on the point cloud trajectory and the running trajectory; Based on the bounding box in the fused trajectory, the distance between the obstacle to be tested and the electric locomotive is determined, the distance is compared with a preset distance threshold, and different risk areas between the electric locomotive and the obstacle to be tested are determined according to the comparison result; In the fusion trajectory with a preset number of frames, if a fusion trajectory with a target number of frames exists in any of the risk areas, the locomotive is controlled according to the type of the risk area, wherein the preset number of frames is greater than the target number of frames.

[0011] In one embodiment, the preset distance threshold includes a first distance threshold, a second distance threshold, and a third distance threshold; the risk area includes a danger area, an alarm area, and a warning area; comparing the distance with the preset distance threshold, and determining the different risk areas between the electric locomotive and the obstacle to be tested based on the comparison result, includes: The area where the distance is less than or equal to the first distance threshold is defined as the danger zone; The region whose distance is greater than the first distance threshold and less than or equal to the second distance threshold is defined as the alarm region. The region whose distance is greater than the second distance threshold and less than or equal to the third distance threshold is designated as the warning region.

[0012] Secondly, embodiments of this application provide an obstacle detection device for an unmanned electric locomotive, wherein a camera and a lidar are installed at the front end of the locomotive body, and the device includes: The passage space construction module is used to determine the image plane and the scene image located on the image plane based on the camera device, identify the track area in the scene image through a preset detection algorithm, project the point cloud data of the lidar onto the image plane, determine the track point cloud data located in the track area, and construct the passage space of the electric locomotive based on the track point cloud data and the three-dimensional dimensions of the electric locomotive. The obstacle confirmation module is used to cluster the point cloud data that meets the preset height to obtain several point cloud clusters. If the three-dimensional space of the point cloud cluster overlaps with the passage space, the point cloud cluster is regarded as the obstacle to be tested. The trajectory acquisition module is used to acquire the point cloud trajectory of any obstacle to be tested and the running trajectory of the scene image in continuous frame mode within a preset time period. An obstacle detection module is used to project the point cloud trajectory onto the image plane, determine the target region of the point cloud trajectory and the two-dimensional region of the running trajectory in the image plane, and output the obstacle type and the position information of the point cloud cluster in the scene image when the overlap rate of the two-dimensional region and the target region is greater than a preset threshold.

[0013] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the obstacle detection method for unmanned electric locomotives as described in the first aspect above.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the obstacle detection method for unmanned electric locomotives as described in the first aspect above.

[0015] The obstacle detection method and apparatus for unmanned electric locomotives provided in this application have at least the following technical effects: Scene images provide rich scene information, while point cloud data provides precise geometric and 3D information. The passage space constructed using scene images, LiDAR point cloud data, and the 3D dimensions of the locomotive can adapt to frequently changing environments and provide accurate passage space for the locomotive. Within the passage space, by clustering point clouds, obstacles are identified based on the overlap between the 3D space of the point cloud clusters and the passage space. This 3D perspective determines whether an obstacle has entered the passage space, reducing misjudgments of fixed targets such as trackside facilities and walls, thus improving obstacle detection accuracy. The point cloud trajectory of the obstacle is projected onto the image plane, and the overlap rate between the point cloud trajectory and the running trajectory is determined. When the overlap rate is greater than a preset threshold, it indicates that the point cloud trajectory and the running trajectory are essentially identical, meaning that both the LiDAR and camera equipment have detected the same obstacle. Through joint verification of the point cloud trajectory and the running trajectory, the running trajectory of the obstacle can still be identified even in complex environments, ensuring accurate obstacle identification and position perception even in complex conditions.

[0016] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating an obstacle detection method for an autonomous electric locomotive according to an exemplary embodiment; Figure 2 This is a schematic diagram illustrating hierarchical control according to an exemplary embodiment; Figure 3 This is a flowchart illustrating an obstacle detection method for an unmanned electric locomotive according to another exemplary embodiment; Figure 4 This is a schematic diagram of an obstacle detection device for an unmanned electric locomotive, according to an exemplary embodiment. Figure 5 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0019] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0020] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0021] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] Electric locomotives are electric traction vehicles used in enclosed industrial and mining rail transport scenarios. In confined spaces such as underground coal mines, metallurgical plants, and tunnel construction sites, electric locomotives rely on batteries or overhead power lines for power supply to pull mining cars or material carriages to achieve cargo transportation.

[0023] The electric locomotive of this application is equipped with a camera and a lidar at its front end. The camera is used to capture images of the scene ahead of the locomotive along its travel path. The lidar is used to acquire point cloud data of the scene ahead of the locomotive along its travel path.

[0024] Based on the above, this application provides an obstacle detection method for unmanned electric locomotives.

[0025] In a first aspect, embodiments of this application provide an obstacle detection method for unmanned electric locomotives, wherein, Figure 1 This is a flowchart illustrating an obstacle detection method for an autonomous electric locomotive according to an exemplary embodiment, such as... Figure 1 As shown, obstacle detection methods for driverless electric locomotives include: Step S101: Based on the camera device, determine the image plane and the scene image located on the image plane. Identify the track area in the scene image through a preset detection algorithm. Project the point cloud data of the lidar onto the image plane to determine the track point cloud data located in the track area. Construct the passage space of the electric locomotive based on the track point cloud data and the three-dimensional dimensions of the locomotive.

[0026] The image plane is determined based on the intrinsic parameters of the camera equipment. When the locomotive travels on the track, the camera equipment located at the front of the locomotive captures an image of the scene ahead in the direction of travel. A preset detection algorithm identifies the track edges in the scene image, and the track region is determined based on these edges. If the scene image shows a track junction, all detected track edges are used, and the distance between any two track edges is obtained. If the distance between the track edges meets a preset edge distance, then different track regions within the track junction are determined.

[0027] The lidar uses the sensor itself as the origin, the lidar's forward direction as the positive x-axis, the perpendicular rightward direction as the positive y-axis, and the upward direction perpendicular to the xy-plane as the positive z-axis, constructing a lidar coordinate system. Point cloud data of the scene ahead along the locomotive's path is acquired within this lidar coordinate system.

[0028] It should be noted that when the camera captures scene images and the LiDAR acquires point cloud data, the scene images and point cloud data are synchronized via the PTP protocol, ensuring that the two data are processed at the same time in subsequent processing.

[0029] A camera coordinate system is constructed with the optical center of the camera lens as the origin, the camera's travel direction as the positive x-axis, the rightward perpendicular x-axis as the positive y-axis, and the upward perpendicular to the xy-plane as the positive z-axis. The intrinsic parameters of the camera are obtained, as well as the extrinsic parameters between the LiDAR coordinate system and the camera coordinate system. The extrinsic parameters include a rotation matrix and a translation vector, denoted as R, t, and K respectively.

[0030] The coordinates of the point cloud data are transformed from the LiDAR coordinate system to the camera coordinate system through coordinate system transformation (R*P+t). Based on the intrinsic parameters of the camera device, the 3D coordinates of the point cloud data in the camera coordinate system are projected onto the image plane through the coordinate transformation relationship (λp=λK(R*P+t)), thus obtaining the image coordinates of the point cloud data in the image plane. Here, R is the rotation matrix, t is the translation vector, p is the image coordinate in the camera coordinate system, P is the coordinate of the point cloud data in the LiDAR coordinate system, and λ is the transformation parameter.

[0031] Based on the image coordinates of the point cloud data, the image coordinates within the orbital region are determined. The purpose of projecting the point cloud data onto the image plane is to filter the point cloud data within the orbital region based on the image coordinates. Therefore, the orbital point cloud data located within the orbital region is determined through the point cloud data mapped from the image coordinates.

[0032] Continuing with step S101, a three-dimensional coordinate system is constructed with the locomotive's front as the origin of the three-dimensional coordinate system, the locomotive's forward direction as the positive x-axis, the direction perpendicular to the x-axis to the right as the positive y-axis, and the direction perpendicular to the xy plane upward as the positive z-axis. The spatial boundaries are then determined based on the three-dimensional coordinate system and the three-dimensional dimensions.

[0033] Based on the three-dimensional dimensions of the locomotive and the trailer it pulls, the spatial limits for the normal passage of the locomotive and trailer are determined. These spatial limits ensure the smooth passage of the locomotive and trailer.

[0034] The x-axis direction is updated based on the orbit point cloud data, and the orbit is divided into different segments along the x-axis at preset intervals.

[0035] The entire track point cloud data is analyzed to determine the track's orientation. Since the positive direction of the x-axis is the locomotive's forward direction, the square of the x-axis is updated based on the track point cloud data to ensure that the subsequently constructed passageway can meet the track's orientation in real time. The x-axis is divided into several segments according to a preset interval. Optionally, the preset interval includes 1 meter or 2 meters. Preferably, the preset interval is 1 meter.

[0036] For any given segment, the center coordinates are determined based on all the track point cloud data within the segment. Then, the passage space for the segment is constructed based on the center coordinates and spatial boundaries.

[0037] Based on the x-axis coordinates of the track point cloud data, the segment into which the track point cloud data falls can be determined. Within any segment, the center coordinates of the track point cloud data within that segment are determined based on all track point cloud data within that segment. This is achieved by calculating the arithmetic mean of the y-axis and z-axis coordinates of all track point cloud data within the segment, and using this arithmetic mean as the center coordinates. Since segments are divided according to the track's orientation, the direction of travel within the passable area can be determined using the y-axis and z-axis coordinates. Based on the center coordinates and spatial boundaries, the travel space for the current segment is constructed. The travel spaces of all segments constitute the travel space for the locomotive's direction of travel. The travel space of each segment satisfies the following formula: in, It is the passage space of the i-th segment. It is half the width of the track boundary. It is the height of the orbital boundary, and (x, y, z) is any spatial coordinate in the passage space of the i-th segment. It is the y-coordinate of the i-th center coordinate. It is the z-axis coordinate of the center coordinate. It is the left endpoint of the i-th segment. It is the right endpoint of the i-th segment.

[0038] Continuing with step S101, spatial boundaries are initially determined in a three-dimensional coordinate system based on the three-dimensional scale of the locomotive and its cargo bed to ensure normal operation and passage. Within each segment, the passage space is determined based on the spatial boundaries and center coordinates. The center coordinates, determined using track point cloud data, represent the offset relative to the ideal straight track in the y-axis direction and the height change in the z-axis, thus accurately determining the track orientation within each segment. Based on the track orientation and spatial boundaries, a precise passage space can be constructed within that segment, enabling accurate obstacle detection in subsequent scenarios. Furthermore, the x-axis is divided into different segments at different preset intervals, effectively discretizing the space the locomotive will pass through. For each precisely constructed passage space, changes in the passage space within each segment and subsequent obstacle detection can be quickly assessed, improving the efficiency of obstacle detection during locomotive operation.

[0039] Step S102: Cluster the point cloud data that meets the preset height to obtain several point cloud clusters. If the three-dimensional space of the point cloud cluster overlaps with the passage space, the point cloud cluster is used as the obstacle to be tested.

[0040] Clustering is performed on all point cloud data that meet a preset height to obtain several point cloud clusters. The three-dimensional space of each point cloud cluster is determined in a three-dimensional coordinate system, where the three-dimensional space includes the horizontal extent of the cluster along the x-axis, the vertical extent along the y-axis, and the vertical extent along the z-axis. If the three-dimensional space of a point cloud cluster overlaps with any passageway, that is, if the boundary of the point cloud cluster overlaps with the boundary of the passageway, then the point cloud cluster is considered an obstacle to be measured. The specific steps for considering a point cloud cluster as an obstacle to be measured include: Step S121: Fit the track plane based on the track point cloud data. In the vertical direction of the track plane, filter the point cloud data that meets the preset height to obtain intermediate point cloud data. The track plane is located in the xy plane of the three-dimensional coordinate system.

[0041] The RANSAC algorithm is used to fit a track plane to all track point cloud data within the track area, where the track plane lies in the xy plane of a 3D coordinate system. The coordinate parameters of the track plane are determined within this plane. Point cloud data located above the track plane and meeting a preset height are selected and used as intermediate point cloud data. Optionally, the preset height is between 10cm and 30cm. The selection of the preset height can be determined based on the type of obstacles present in the scene. If the obstacles in the scene where the locomotive is moving are small falling rocks, the preset height can be increased, for example, the preset distance range could be 10cm-40cm.

[0042] Step S122: Cluster the intermediate point cloud data to obtain several point cloud clusters, determine the three-dimensional space of each point cloud cluster, and determine the passage space of the point cloud cluster based on the segment of the three-dimensional space located on the x-axis.

[0043] Using a neighborhood radius of 0.3 meters and a minimum number of points in the neighborhood of 6, the DBSCAN clustering algorithm was used to cluster the intermediate point cloud data to obtain several point cloud clusters. Each point cloud cluster is discrete, and the three-dimensional space of each point cloud is determined. The three-dimensional space includes the horizontal range of the point cloud cluster in the x-axis direction, the vertical range in the y-axis direction, and the vertical range in the z-axis direction.

[0044] Specifically, three-dimensional space satisfies: ,in, It is the minimum x-coordinate of the point cloud cluster in the three-dimensional space along the x-axis. It is the maximum x-coordinate of the point cloud cluster in the three-dimensional space along the x-axis; It is the minimum y-coordinate of the point cloud cluster in the 3D space along the y-axis. It is the maximum value of the y-coordinate of the point cloud cluster in the 3D space along the y-axis; It is the minimum vertical coordinate of the point cloud cluster in the z-axis direction in the three-dimensional space. It is the maximum vertical coordinate of the point cloud cluster in the z-axis direction in the three-dimensional space.

[0045] Based on the lateral extent of the point cloud cluster in the 3D space along the x-axis, the segment of the x-axis where the point cloud cluster is located is determined. The passageway where the point cloud cluster is located is then determined based on this segment. If the minimum x-coordinate of the point cloud cluster is greater than a preset target distance, then the point cloud cluster is not processed. Optionally, the preset target distance is 30 meters.

[0046] Step S123: If the three-dimensional space of the point cloud cluster overlaps with the passage space in both the y-axis and z-axis directions, then the point cloud cluster is considered as an obstacle to be tested.

[0047] In the passage space, the x-axis horizontal coordinate is used to determine the location of different segments. The overlap between the 3D space of the point cloud cluster and the passage space is determined by the y-axis vertical coordinate and the z-axis vertical coordinate. If the 3D space of the point cloud cluster overlaps with the passage space in both the y-axis and z-axis directions, the point cloud cluster is considered an obstacle to be measured. Specifically, the 3D space of the point cloud cluster overlaps with the passage space in both the y-axis and z-axis directions, meaning that the vertical range of the 3D space of the point cloud cluster in the y-axis direction overlaps with the vertical range of the passage space in the y-axis direction, and the vertical range of the 3D space of the point cloud cluster in the z-axis direction overlaps with the vertical range of the passage space in the z-axis direction, specifically satisfying the following formula: in, It is the minimum y-coordinate of the point cloud cluster in the 3D space along the y-axis. It is the maximum value of the y-coordinate of the point cloud cluster in the 3D space along the y-axis. It is the y-coordinate of the i-th center coordinate. It is half the width of the track boundary; It is the minimum vertical coordinate of the point cloud cluster in the z-axis direction in the three-dimensional space. It is the maximum vertical coordinate of the point cloud cluster in the z-axis direction in the three-dimensional space. It is the height of the orbital boundary.

[0048] Continuing with step S102, the point cloud data located vertically upward on the track plane is filtered to remove the influence of the track point cloud, and the intermediate point cloud data is clustered to obtain point cloud clusters. The point cloud clusters are then compared with the passage space to determine whether they are obstacles to be tested. Judging the relationship between obstacles and passage space from the perspective of three-dimensional space can effectively avoid misjudgments caused by non-obstacle targets such as trackside facilities and walls, and achieve a preliminary judgment of obstacles, thereby improving the accuracy of subsequent obstacle judgment.

[0049] Step S103: For any obstacle to be tested, acquire the point cloud trajectory of the obstacle and the running trajectory of the scene image in continuous frame mode within a preset time period.

[0050] Multiple obstacles may exist in the passageway. For each obstacle, its point cloud trajectory and running trajectory are acquired in the same way. For any obstacle, its point cloud data is acquired within a preset time period. In the continuous frame scene images within the preset time period, the obstacle is identified using an object detection algorithm, and its running trajectory in the continuous frames is determined.

[0051] The steps for obtaining the point cloud trajectory include: Based on point cloud clusters within the passage space, geometric features of the point cloud clusters are obtained. These features include the centroid, the bounding box of the point cloud cluster, and the nearest point of the point cloud cluster. The nearest point of the point cloud cluster is the point cloud data with the minimum distance to the locomotive within the cluster. The geometric features satisfy the following formula: , in, It is the centroid of point cloud cluster j in frame t. These are the x, y, and vertices of the centroid of point cloud cluster j in frame t, respectively. It is the bounding box of point cloud cluster j in frame t. It is the horizontal range on the x-axis of the bounding box of the point cloud cluster. It is the vertical range on the y-axis of the bounding box of the point cloud cluster. It is the vertical range on the z-axis of the bounding box of the point cloud cluster.

[0052] Within a preset time period, the centroid is updated temporally using a preset filtering algorithm to obtain the point cloud trajectory.

[0053] Using the centroid of the point cloud cluster as the reference point for its movement, the centroid is updated temporally using a Kalman filter algorithm within a preset time period to obtain the point cloud trajectory. The point cloud trajectory satisfies the following formula: in, This is the point cloud trajectory state in frame t. This represents the running state of point cloud cluster j in frame t. It is the centroid of point cloud cluster j in frame t. It is the bounding box of point cloud cluster j in frame t. It is the velocity of point cloud cluster j in frame t.

[0054] Continuing with step S103, in the continuous frame scene images within a preset time period, the obstacle to be tested in each frame scene image is determined using an object detection algorithm. During object detection, a target detection box for the obstacle to be tested is determined in each frame. The trajectory of the obstacle to be tested is determined based on the target detection boxes in the continuous frames. The object detection algorithm includes identifying the obstacle to be tested and determining its position; the process of identifying the obstacle to be tested includes identifying its category. The trajectory then satisfies the following formula: in, It is the trajectory of the i-th obstacle to be tested in the t-th frame. Let i be the bounding box of the i-th obstacle to be tested in frame t. The category of the obstacle to be tested. , where is the confidence level.

[0055] It should be noted that since the steps performed for each obstacle to be tested are identical, the detection method in step 103 is for a single obstacle to be tested. However, when there are multiple point cloud clusters and multiple obstacles to be tested in the scene image, the detection is performed simultaneously for multiple targets. That is, multi-target detection is performed on the obstacles to be tested in consecutive frames of scene image, while the point cloud trajectory of each point cloud cluster is determined simultaneously in multiple point cloud clusters.

[0056] The point cloud trajectory of the point cloud cluster and the running trajectory of the scene image are obtained so that the dynamic trajectory can be jointly verified in the future to achieve dynamic detection of the obstacle to be tested.

[0057] Step S104: Project the point cloud trajectory onto the image plane, and determine the target area and the two-dimensional area of ​​the running trajectory in the image plane. When the overlap rate between the two-dimensional area and the target area is greater than a preset threshold, output the obstacle type and the position information of the point cloud cluster in the scene image.

[0058] Based on the extrinsic parameters of the projection device and LiDAR, the point cloud trajectory of the point cloud cluster is projected onto the image plane. The point cloud trajectory is the movement path of the point cloud cluster in different frames within a preset time period. The target region of the point cloud trajectory is determined based on the area occupied by the 2D image of each point cloud cluster, and the 2D region of the trajectory is determined based on the bounding box of each obstacle to be detected in each frame of the scene image. The intersection and union of the 2D region and the target region are determined, and the overlap rate is determined based on the ratio of the intersection to the union. When the overlap rate is greater than a preset threshold, the obstacle type and point cloud cluster position information obtained during target detection in the scene image are output. The position information of the point cloud cluster is determined by the geometric features of each point cloud cluster. If the overlap rate is less than or equal to the preset threshold, it indicates that the currently identified obstacle to be detected is a false detection. The obstacle types include falling rocks, electric locomotives, truck beds, and personnel.

[0059] More specifically, target detection is performed on the obstacles to be tested in consecutive frame scene images to determine the obstacle type. Based on the geometric features of each point cloud cluster in the point cloud trajectory, the location information of the obstacle to be tested is determined. That is, the obstacle type is included in the running trajectory and the location information is included in the point cloud trajectory.

[0060] Using the extrinsic parameters of the camera equipment and LiDAR, the point cloud trajectory is projected onto the image plane. In the image plane, the target region of each point cloud cluster in the point cloud trajectory is determined based on the centroid and bounding box. The bounding box of the obstacle to be tested in each frame of the scene image is obtained through the target detection algorithm to determine the two-dimensional region of the obstacle's trajectory.

[0061] Obtain the intersection and union of the target region and the two-dimensional region. Determine the overlap rate based on the ratio of the intersection to the union. When the overlap rate is greater than a preset threshold, output the obstacle type and the location information of the point cloud cluster.

[0062] When the overlap rate exceeds a preset threshold, it indicates that the trajectory of the obstacle identified in the scene image highly overlaps with the point cloud trajectory determined by the point cloud data. This means the obstacle has been identified by both the LiDAR and camera equipment, and accurate obstacle detection is achieved through joint verification of the LiDAR and camera equipment. Once it is determined that the obstacle is an actual obstacle, the system outputs the obstacle type determined during target detection in the scene image, as well as its position information within the point cloud cluster. The electric locomotive can then respond promptly based on the obstacle type and position information to ensure its safety during operation.

[0063] When the overlap rate is less than or equal to a preset threshold, it indicates a significant difference between the trajectory of the obstacle identified in the scene image and the point cloud trajectory determined by the point cloud data. This means that either the LiDAR or the camera has made a recognition error, and therefore the obstacle cannot be identified as an actual obstacle. However, since the LiDAR acquires point cloud data along the locomotive's direction of travel, the point cloud trajectory is retained when the overlap rate is less than or equal to the preset threshold for further safety braking of the locomotive.

[0064] Continuing with step S104, the point cloud trajectory is projected onto the image plane. The target area and two-dimensional determination of the running trajectory of the point cloud trajectory are established in the image plane. The overlap rate between the target area and the two-dimensional area determines whether the obstacle to be detected is a real obstacle. Through joint verification by the lidar and camera equipment, accurate detection of actual obstacles is achieved. After confirming the obstacle as an actual obstacle, the type and location information of the obstacle are output to provide a basis for the subsequent response strategy of the electric locomotive. Furthermore, lidar is less affected by environmental factors. The joint verification of lidar and camera equipment compensates for the low detection accuracy of camera equipment due to environmental factors. Therefore, the joint verification method can still maintain stable and accurate judgment of obstacles in scenarios such as changes in lighting, dust, and low light, improving the stability and reliability of obstacle detection.

[0065] After outputting the obstacle type and point cloud cluster location information from the scene image, the electric locomotive responds based on the obstacle type and location information. Specifically, this includes: Step S141: When the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, a fused trajectory is generated based on the point cloud trajectory and the running trajectory. The fused trajectory includes the obstacle type of the running trajectory and the geometric features of the point cloud trajectory. Optionally, the method for generating the fused trajectory includes: stitching the point cloud trajectory and the running trajectory together to obtain the fused trajectory; fusing the state vectors of the point cloud trajectory and the running trajectory through extended Kalman filtering or particle filtering to generate a smooth and stable fused trajectory; and encoding the image features of the running trajectory and the point cloud features of the point cloud trajectory through a neural network algorithm to generate the fused trajectory.

[0066] Step S142: Based on the bounding box in the fused trajectory, determine the distance between the obstacle to be tested and the electric locomotive, compare the distance with a preset distance threshold, and determine the different risk areas between the electric locomotive and the obstacle to be tested based on the comparison results.

[0067] Based on bounding boxes in the fused trajectory, the shortest distance between the obstacle and the locomotive is determined. Based on this shortest distance, the locomotive is positioned in different risk zones to ensure its safe operation. Specifically, obtaining the shortest distance between the locomotive and the obstacle satisfies the following conditions: Based on the bounding box of the fused trajectory, the shortest distance between the obstacle to be measured and the electric locomotive in the x-axis direction is determined. The shortest distance specifically satisfies the following formula: in, It is the shortest distance between the obstacle to be measured and the electric locomotive along the x-axis. It is the x-axis coordinate of the bounding box of the obstacle to be measured, which is closest to the locomotive in the x-axis direction.

[0068] Because locomotives vibrate during operation, the calculated shortest distance needs to be smoothed to reduce errors caused by equipment vibration. The smoothed distance is obtained by smoothing the shortest distance using the following formula: in, It is the smoothed distance at time t. It is the position weight of the previous moment. It is the smoothed distance at time t-1. It is the closest distance between the obstacle to be measured and the electric locomotive in the x-axis direction.

[0069] After determining the smoothing distance, the relative speed between the obstacle and the locomotive and the estimated collision time can also be determined based on the smoothing distance, thus providing a basis for the locomotive to respond safely. Specifically, the following formula applies: , in, It is the relative velocity between the obstacle and the locomotive at time t. It is the smoothed distance at time t. It is the smoothed distance at time t-1, and Δt is the time difference between time t and time t-1; This is the estimated collision time. It is the velocity of the obstacle at time t. It is the speed threshold.

[0070] Continuing with step S142, the distance is compared with a preset distance threshold. Based on the comparison result, different risk zones between the electric locomotive and the obstacle to be measured are determined. The distances between the electric locomotive and the obstacle are compared using smoothed distances. The preset distance thresholds include a first distance threshold, a second distance threshold, and a third distance threshold. The risk zones include danger zones, alarm zones, and warning zones. Regions with a smoothing distance less than or equal to the first distance threshold are designated as danger zones. Regions with a smoothing distance greater than the first distance threshold and less than or equal to the second distance threshold are designated as alarm zones. Regions with a smoothing distance greater than the second distance threshold and less than or equal to the third distance threshold are designated as warning zones.

[0071] Optionally, the sensing distance of the electric locomotive is set to 30 meters, the first distance threshold is 30 meters, the second distance threshold is 20 meters, and the third distance threshold is 10 meters. Within 30 meters of the electric locomotive's direction of travel, areas where the smooth distance between the obstacle and the electric locomotive is less than or equal to 10 meters are designated as danger zones, areas where the smooth distance between the obstacle and the electric locomotive is greater than 10 meters but less than or equal to 20 meters are designated as alarm zones, and areas where the smooth distance between the obstacle and the electric locomotive is greater than 20 meters but less than or equal to 30 meters are designated as warning zones.

[0072] Step S143: In the fusion trajectory of the preset number of frames, if there is a fusion trajectory of the target number of frames in any risk area, then control the locomotive according to the type of risk area, wherein the preset number of frames is greater than the target number of frames.

[0073] After determining that the 3D space of the obstacle to be detected overlaps with the passage space, i.e., the obstacle encroaches on the passage space, a preset number of frames for the fusion trajectory is determined through a sliding window. Within the preset number of fusion trajectories, based on the risk area where the fusion trajectory is located, if a fusion trajectory of the target number of frames exists within the risk area, then the fusion trajectory is determined to be located within the current risk area. This avoids detection errors caused by false detections in a single frame and improves the accuracy of obstacle detection. The locomotive is then controlled to respond according to the type of risk area.

[0074] The fusion trajectory that determines the number of target frames within the risk area satisfies the following formula: in, It is the number of frames that satisfy the condition at time t. yes The smoothing distance at any given time, N is the preset number of frames for the sliding window, M is the trigger threshold, Zone() is the region determination function, Z is the danger zone type, and M is the trigger threshold. It is an indicator function that outputs 1 if the content inside the parentheses is true, and 0 if the content inside the parentheses is false.

[0075] In other words, if the three-dimensional space of the obstacle to be tested overlaps with the passage space, and there is a fused trajectory of consecutive frames within the risk area, then it can be determined that the obstacle exists in the passage area ahead of the locomotive's direction of travel. Based on the location of the obstacle in the danger zone, the locomotive can be subject to graded control according to the type of danger zone.

[0076] Specifically, hierarchical control satisfies the following: If the fusion trajectory is located in the warning area, that is, if the obstacle is located in the warning area and there is a large distance between the obstacle and the locomotive, the locomotive will issue an audible and visual alarm and report the type and location information of the obstacle to the control system.

[0077] If the fused trajectory is located in the alarm area, that is, if the obstacle is located in the alarm area and the obstacle is close to the locomotive, the locomotive will execute a deceleration command and continuously update the smooth distance between the obstacle and the locomotive to adjust the speed of the locomotive in real time, so as to avoid the locomotive from colliding with the obstacle and allow the locomotive to pass safely.

[0078] If the fusion trajectory is located in a danger zone, that is, if the obstacle is located in a danger zone and the distance between the obstacle and the locomotive is very close, the locomotive will trigger an emergency braking command to avoid a collision between the locomotive and the obstacle.

[0079] In one embodiment, Figure 2 This is a schematic diagram illustrating hierarchical control according to an exemplary embodiment, such as... Figure 2 As shown, when the obstacle is less than 10 meters away, it is located in a danger zone, and the locomotive applies emergency braking. When the obstacle is more than 10 meters away but less than or equal to 20 meters away, it is located in a warning zone, and the locomotive decelerates. When the obstacle is more than 20 meters away but less than or equal to 30 meters away, it is located in an alarm zone, and an audible and visual warning is issued.

[0080] In addition to meeting the hierarchical control requirements, the locomotive's response also meets the safety finite principle. That is, when there are multiple obstacles in the passage space, the nearest obstacle is selected as the main control object, and a safety response is carried out according to the danger zone where the main control object is located.

[0081] It should be noted that if the overlap rate between the point cloud trajectory and the running trajectory is less than or equal to a preset threshold, it is judged as a false detection of the obstacle to be tested. However, the point cloud trajectory is still retained. If the point cloud data determines that the three-dimensional space of the obstacle to be tested overlaps with the passage space, and the overlap degree meets 80%, that is, the obstacle encroaches on the passage space, even if it is judged as a false detection of the obstacle to be tested, the locomotive response will still be executed according to the danger zone where the obstacle to be tested is located, in order to ensure the safety of the locomotive.

[0082] Referring to steps S141 to S143, the system is divided into warning areas, alarm areas, and danger areas based on the smoothing distance. Different safety responses are executed in different danger areas according to different fusion trajectories, including audible and visual alarms, deceleration, and emergency braking. By judging whether the fusion trajectory meets the target number of frames in the danger area, the accuracy of the fusion trajectory is judged from the perspective of temporal continuity, so as to ensure accurate judgment of obstacles, avoid erroneous judgments caused by single-frame false detection, improve the safety of the unmanned electric locomotive, and reduce the risk of collision between the electric locomotive and obstacles.

[0083] In one embodiment, Figure 3 This is a flowchart illustrating an obstacle detection method for an unmanned electric locomotive according to another exemplary embodiment, such as... Figure 3 As shown, image data of the scene is acquired through camera equipment, and track lines are extracted based on the image data to determine the track region. Point cloud data is obtained through LiDAR, and a three-dimensional forward clearance range, i.e., the passage space, is constructed based on the point cloud data of the track region and the track region. The LiDAR point cloud data is also used for clustering to obtain point cloud clusters. Image data is also used for target detection and tracking. Obstacles are determined based on the target tracking results and the point cloud clustering results within the passage space.

[0084] In summary, the passage space of each segment is determined based on spatial boundaries and center coordinates. The center coordinates represent the offset relative to the ideal straight track in the y-axis direction and the height change in the z-axis within the segment, thus accurately determining the track orientation within each segment. Constraints based on track orientation and spatial boundaries allow for the construction of a precise passage space within that segment. Within this precise passage space, clustered point cloud data identifies obstacles to be tested. The relationship between these obstacles and the passage space in three-dimensional space effectively avoids misjudgments caused by non-obstacle targets such as trackside facilities and walls, achieving accurate obstacle identification. In the dynamic trajectory of the obstacle to be tested, the synchronous verification of the point cloud trajectory and the running trajectory by the camera equipment and LiDAR ensures accurate detection of actual obstacles. Based on the distance judgment between the obstacle and the locomotive, danger zones are delineated. By fusing the fusion trajectory of the number of target frames within the danger zone, the temporal judgment of the fusion trajectory avoids misjudgments in single frames. Furthermore, the locomotive executes a safety response based on its location within the danger zone, improving the safety of the unmanned locomotive's operation.

[0085] Secondly, embodiments of this application provide an obstacle detection device for an unmanned electric locomotive. The front of the locomotive is equipped with a camera and a lidar sensor. Figure 4 This is a schematic diagram of an obstacle detection device for an unmanned electric locomotive, according to an exemplary embodiment. Figure 4 As shown, the obstacle detection device for driverless electric locomotives includes: The passage space construction module is used to determine the image plane and the captured scene image based on the camera equipment, identify the track area through a preset detection algorithm, project the point cloud data of the lidar onto the image plane, determine the track point cloud data located within the track area, and construct the passage space of the electric locomotive based on the track point cloud data and the three-dimensional dimensions of the electric locomotive. The obstacle identification module is used to cluster point cloud data to obtain several point cloud clusters. If the three-dimensional space of the point cloud cluster overlaps with the passage space, the point cloud cluster is regarded as the obstacle to be tested. The trajectory acquisition module is used to acquire the point cloud trajectory and running trajectory in continuous frame scene images of any obstacle to be tested within a preset time period. The obstacle detection module projects the point cloud trajectory onto the image plane, determines the target area and the two-dimensional area of ​​the running trajectory in the image plane, and outputs the obstacle type and point cloud cluster position information of the scene image when the overlap rate between the two-dimensional area and the target area is greater than a preset threshold.

[0086] In summary, the obstacle detection device for unmanned electric locomotives provided in this application obtains target insulating oil from a foil capacitor. This target insulating oil contains gases generated during the use of the foil capacitor. The gas concentration in the target insulating oil is obtained using a colorimetric chromatograph, and this gas concentration is input into a fault detection model to determine the fault result of the foil capacitor. This non-destructive sampling method obtains the gas in the foil capacitor, and through the mapping relationship between gas concentration and fault type in the fault detection model, it achieves accurate assessment of the damage state of the foil capacitor. It should be noted that the obstacle detection device for unmanned electric locomotives provided in this embodiment is used to implement the above-described embodiments, and details already described will not be repeated. As used above, terms such as "module," "unit," and "subunit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the above embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0087] Thirdly, embodiments of this application provide an electronic device, Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. (e.g.) Figure 5 As shown, the electronic device may include a processor 81 and a memory 82 storing computer program instructions.

[0088] Specifically, the processor 81 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0089] The memory 82 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include removable or non-removable (or fixed) media. Where appropriate, the memory 82 may be internal or external to a data processing device. In a particular embodiment, the memory 82 is non-volatile memory. In a particular embodiment, the memory 82 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0090] The memory 82 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 81.

[0091] The processor 81 reads and executes computer program instructions stored in the memory 82 to implement any of the obstacle detection methods for unmanned electric locomotives in the above embodiments.

[0092] In one embodiment, the obstacle detection device for an unmanned electric locomotive may further include a communication interface 83 and a bus 80. Wherein, as Figure 5 As shown, the processor 81, memory 82, and communication interface 83 are connected through bus 80 and complete communication with each other.

[0093] The communication interface 83 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication interface 83 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0094] Bus 80 includes hardware, software, or both, that couples together components of an obstacle detection device for an unmanned electric locomotive. Bus 80 includes, but is not limited to, at least one of the following: Data Bus, Address Bus, Control Bus, Expansion Bus, and Local Bus. For example, and not as a limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0095] Fourthly, embodiments of this application provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the obstacle detection method for unmanned electric locomotives provided in the first aspect.

[0096] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0097] In a possible implementation, the present invention can also be implemented as a program product comprising program code, which, when the program product is run on a terminal device, causes the terminal device to perform steps implementing the obstacle detection method for unmanned electric locomotives provided in the first aspect.

[0098] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An obstacle detection method for unmanned electric locomotives, characterized in that, The locomotive is equipped with a camera and a lidar at the front of its body, and the method includes: Based on the camera device, an image plane and a scene image located on the image plane are determined. A preset detection algorithm is used to identify the track region in the scene image. The point cloud data of the lidar is projected onto the image plane to determine the track point cloud data located within the track region. Based on the track point cloud data and the three-dimensional dimensions of the electric locomotive, the passage space of the electric locomotive is constructed. Clustering the point cloud data that meets the preset height yields several point cloud clusters. If the three-dimensional space of the point cloud cluster overlaps with the passage space, the point cloud cluster is used as an obstacle to be tested. For any of the obstacles to be tested, the point cloud trajectory of the obstacle to be tested and the running trajectory of the scene image in continuous frame mode are acquired within a preset time period. The point cloud trajectory is projected onto the image plane, and the target region of the point cloud trajectory and the two-dimensional region of the running trajectory are determined in the image plane. When the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, the obstacle type of the scene image and the position information of the point cloud cluster are output.

2. The obstacle detection method for unmanned electric locomotives according to claim 1, characterized in that, The construction of the locomotive's passage space based on the track point cloud data and the locomotive's three-dimensional dimensions includes: A three-dimensional coordinate system is constructed with the front of the electric locomotive as the origin of the three-dimensional coordinate system, the forward direction of the electric locomotive as the positive x-axis, the rightward direction perpendicular to the x-axis as the positive y-axis, and the upward direction perpendicular to the xy plane as the positive z-axis. The spatial boundaries are determined based on the three-dimensional coordinate system and the three-dimensional dimensions. The direction of the x-axis is updated based on the orbit point cloud data, and the orbit is divided into different segments along the x-axis at preset intervals. For any given segment, the center coordinates of the track in the three-dimensional coordinate system are determined based on all the track point cloud data within the segment. The passage space of the segment is then constructed based on the center coordinates and the spatial boundaries.

3. The obstacle detection method for unmanned electric locomotives according to claim 2, characterized in that, The clustering of point cloud data that satisfies a preset height yields several point cloud clusters. If the three-dimensional space of the point cloud cluster overlaps with the passage space, the point cloud cluster is used as an obstacle to be measured, including: Based on the orbit point cloud data, a orbit plane is fitted. In the vertical direction of the orbit plane, the point cloud data that meets the preset height is filtered to obtain intermediate point cloud data. The orbit plane is located in the xy plane of the three-dimensional coordinate system. Clustering the intermediate point cloud data yields several point cloud clusters, determining the three-dimensional space of each point cloud cluster, and based on the segment of the three-dimensional space located on the x-axis, determining the passage space where the point cloud cluster is located; If the three-dimensional space of the point cloud cluster overlaps with the passage space in both the y-axis and z-axis directions, then the point cloud cluster is considered as the obstacle to be measured.

4. The obstacle detection method for unmanned electric locomotives according to claim 1, characterized in that, The step of acquiring the point cloud trajectory of the obstacle to be tested within a preset time period includes: Based on the point cloud clusters within the passage space, the geometric features of the point cloud clusters are obtained, and the geometric features include the centroid; Within the preset time period, the centroid is updated temporally using a preset filtering algorithm to obtain the point cloud trajectory.

5. The obstacle detection method for unmanned electric locomotives according to claim 4, characterized in that, The geometric features also include the bounding box of the point cloud cluster. The point cloud trajectory is projected onto the image plane, and a target region and a two-dimensional region of the trajectory are determined in the image plane. When the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, the obstacle type and the position information of the point cloud cluster in the scene image are output, including: Target detection is performed on the scene images in consecutive frames to determine the obstacle type of the obstacle, and the position information of the obstacle is determined based on the geometric features of each point cloud cluster in the point cloud trajectory. The point cloud trajectory is projected onto the image plane, and the target region of each point cloud cluster in the point cloud trajectory is determined according to the bounding box in the image plane. The two-dimensional region of the obstacle to be tested in each frame of the scene image is determined by the target detection algorithm. Obtain the intersection and union of the target region and the two-dimensional region, determine the overlap rate based on the ratio of the intersection to the union, and output the obstacle type and the location information of the point cloud cluster when the overlap rate is greater than the preset threshold.

6. The obstacle detection method for unmanned electric locomotives according to claim 5, characterized in that, After outputting the obstacle type and point cloud cluster location information of the scene image when the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, the method further includes: When the overlap rate between the two-dimensional region and the target region is greater than a preset threshold, a fused trajectory is generated based on the point cloud trajectory and the running trajectory; Based on the bounding box in the fused trajectory, the distance between the obstacle to be tested and the electric locomotive is determined, the distance is compared with a preset distance threshold, and different risk areas between the electric locomotive and the obstacle to be tested are determined according to the comparison result; In the fusion trajectory with a preset number of frames, if a fusion trajectory with a target number of frames exists in any of the risk areas, the locomotive is controlled according to the type of the risk area, wherein the preset number of frames is greater than the target number of frames.

7. The obstacle detection method for unmanned electric locomotives according to claim 6, characterized in that, The preset distance thresholds include a first distance threshold, a second distance threshold, and a third distance threshold. The risk areas include danger zones, alarm zones, and warning zones. The step of comparing the distances with the preset distance thresholds and determining the different risk zones between the electric locomotive and the obstacle to be measured based on the comparison results includes: The area where the distance is less than or equal to the first distance threshold is defined as the danger zone; The region whose distance is greater than the first distance threshold and less than or equal to the second distance threshold is defined as the alarm region. The region whose distance is greater than the second distance threshold and less than or equal to the third distance threshold is designated as the warning region.

8. An obstacle detection device for unmanned electric locomotives, characterized in that, The front of the electric locomotive is equipped with a camera and a lidar, the device comprising: The passage space construction module is used to determine the image plane and the scene image located on the image plane based on the camera device, identify the track area in the scene image through a preset detection algorithm, project the point cloud data of the lidar onto the image plane, determine the track point cloud data located in the track area, and construct the passage space of the electric locomotive based on the track point cloud data and the three-dimensional dimensions of the electric locomotive. The obstacle confirmation module is used to cluster the point cloud data that meets the preset height to obtain several point cloud clusters. If the three-dimensional space of the point cloud cluster overlaps with the passage space, the point cloud cluster is regarded as the obstacle to be tested. The trajectory acquisition module is used to acquire the point cloud trajectory of any obstacle to be tested and the running trajectory of the scene image in continuous frame mode within a preset time period. An obstacle detection module is used to project the point cloud trajectory onto the image plane, determine the target region of the point cloud trajectory and the two-dimensional region of the running trajectory in the image plane, and output the obstacle type and the position information of the point cloud cluster in the scene image when the overlap rate of the two-dimensional region and the target region is greater than a preset threshold.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the obstacle detection method for an unmanned electric locomotive as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the obstacle detection method for unmanned electric locomotives as described in any one of claims 1 to 7.

Images