Obstacle identification method, device and equipment and rail crane

Abstract

The invention provides an obstacle identification method, device and equipment and a rail crane. According to the obstacle recognition method provided by the invention, the problems of target missing detection and space misjudgment in a low-visibility environment in the prior art can be effectively solved by collecting image data from different dimensions (the vehicle head direction and the roadway side direction). Firstly, the first image acquisition device and the second image acquisition device are used for respectively acquiring image data from a vehicle head and a side wall, so that the omnibearing perception of the advancing path of the rail crane is ensured, and the limitation that a traditional monocular vision system cannot comprehensively identify obstacles in a complex environment is overcome. And secondly, through fusion of multi-dimensional image data, the position of the obstacle on the traveling path of the crane can be accurately judged, and the conditions of misjudgment and missing detection are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of rail crane control, and in particular to an obstacle recognition method, device, equipment and a rail crane. Background Art

[0002] Underground monorail cranes are rail-based transportation equipment used within mine tunnels, primarily for transporting materials, equipment, and personnel. They operate on a monorail system installed at the top of the tunnel, with the crane suspended along the track. They are efficient, safe, and flexible, making them a crucial component of modern mine transportation systems.

[0003] In the prior art, during the travel of an underground vehicle, there is a technical problem of inaccurate obstacle recognition in a target area (eg, a stop). Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide an obstacle identification method, device, equipment and rail crane to solve the technical problem in the related art of inaccurate obstacle identification in the target area (for example, a stop point) during the driving of an underground vehicle.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for identifying an obstacle, comprising: When it is detected that the rail crane enters a target area inside the tunnel, at least first-dimensional image data and second-dimensional image data are acquired; wherein the first-dimensional image data is image data acquired by a first image acquisition device, and the second-dimensional image data is image data acquired by a second image acquisition device; wherein the first image acquisition device is an image acquisition device configured at the front of the rail crane for acquiring image data in the direction of travel of the rail crane, and the second image acquisition device is configured at the side of the tunnel in the target area for acquiring image data of the target area; Based on at least the first-dimensional image data and the second-dimensional image data, it is determined whether there is an obstacle on the path of the rail crane.

[0006] Furthermore, a wireless beacon device is provided at the entrance of the target area, and the rail crane is equipped with a radio frequency identification module that matches the wireless beacon device; The step of acquiring at least first-dimensional image data and second-dimensional image data when detecting that the rail crane enters the target area inside the tunnel comprises: When the rail crane travels into the communication range of the wireless beacon device, the rail crane receives the area entry signal sent by the wireless beacon device through the radio frequency identification module; In response to the area entry signal, sending a synchronization start instruction to the first image acquisition device and the second image acquisition device; wherein the synchronization start instruction is used to trigger the first image acquisition device and the second image acquisition device to simultaneously acquire corresponding image data; Image data sent by the first image acquisition device and the second image acquisition device are received respectively, thereby obtaining image data of the first dimension and image data of the second dimension.

[0007] Furthermore, the step of determining whether there is an obstacle on the path of the rail crane based on at least the first-dimensional image data and the second-dimensional image data includes: Performing target detection based on the image data of the first dimension and the image data of the second dimension to obtain a first detected target and a second detected target; wherein the first detected target is a detected target indicated in the image data of the first dimension, and the second detected target is a detected target indicated in the image data of the second dimension; Unifying the first detection target and the second detection target into a path coordinate system based on the traveling direction of the rail crane; Based on the coordinate distribution of each detection target in the path coordinate system, it is determined whether there is an obstacle; wherein the obstacle is a target whose projection point is located within the geometric boundary of the rail crane's travel path and whose real-time distance from the rail crane is less than a safety threshold.

[0008] Furthermore, the X-axis of the path coordinate system extends along the direction of travel of the rail crane, and the Y-axis is a direction perpendicular to the X-axis on the horizontal plane; The step of determining whether there is an obstacle based on the coordinate distribution of each detection target in the path coordinate system includes: Determine whether a target satisfies a first judgment condition; wherein the first judgment condition is that the absolute value of the target's coordinate in the Y-axis direction is less than or equal to the sum of half the track projection width and a preset lateral safety margin; Determine whether a target satisfies a second judgment condition; wherein the second judgment condition is that the coordinate value of the target in the X-axis direction is less than or equal to a safety threshold; wherein the safety threshold is a safety distance threshold dynamically calculated based on the current speed and braking parameters of the rail crane; An object that satisfies both the first judgment condition and the second judgment condition is judged as an obstacle.

[0009] Furthermore, the step of unifying the first detection target and the second detection target into a path coordinate system based on the traveling direction of the rail crane includes: Obtaining first calibration parameters of the first image acquisition device and second calibration parameters of the second image acquisition device; wherein the first calibration parameters include an intrinsic parameter matrix of the first image acquisition device and an extrinsic parameter matrix from the vehicle head coordinate system to the path coordinate system, and the second calibration parameters include an intrinsic parameter matrix of the second image acquisition device and an extrinsic parameter matrix from the side coordinate system to the path coordinate system; Based on the first calibration parameter, converting the two-dimensional pixel coordinates in the image data of the first dimension into three-dimensional coordinates in a path coordinate system; Based on the second calibration parameters, converting the two-dimensional pixel coordinates in the image data of the second dimension into three-dimensional coordinates in a path coordinate system; A preset data fusion method is used to fuse the three-dimensional coordinates obtained by the first coordinate transformation and the second coordinate transformation to generate the final coordinates of each target in the path coordinate system.

[0010] Furthermore, the step of performing target detection based on the image data of the first dimension and the image data of the second dimension to obtain a first detected target and a second detected target includes: Performing preset image preprocessing on the image data of the first dimension and the image data of the second dimension to obtain preprocessed image data of the first dimension and preprocessed image data of the second dimension; Target detection is performed on the preprocessed image data of the first dimension and the preprocessed image data of the second dimension, respectively, to obtain a first detected target and a second detected target.

[0011] Furthermore, the step of performing preset image preprocessing on the image data of the first dimension and the image data of the second dimension to obtain the preprocessed image data of the first dimension and the preprocessed image data of the second dimension includes: A preset image defogging algorithm is executed on the image data of the first dimension and the image data of the second dimension to obtain the image data of the first dimension and the image data of the second dimension after the image defogging.

[0012] In a second aspect, the present invention provides an obstacle recognition module, comprising: an acquisition module configured to acquire at least first-dimensional image data and second-dimensional image data upon detecting that the rail crane enters a target area within a laneway; wherein the first-dimensional image data is image data acquired by a first image acquisition device, and the second-dimensional image data is image data acquired by a second image acquisition device; wherein the first image acquisition device is an image acquisition device disposed at the front of the rail crane for acquiring image data in the direction of travel of the rail crane, and the second image acquisition device is disposed at a laneway sidewall in the target area for acquiring image data of the target area; A determination module is configured to determine whether there is an obstacle on a path of the rail crane based at least on the image data of the first dimension and the image data of the second dimension.

[0013] In a third aspect, the present invention provides an electronic device, comprising: a memory, and one or more processors communicatively coupled to the memory; The memory stores instructions that can be executed by the one or more processors. The instructions are executed by the one or more processors to enable the one or more processors to implement the above method.

[0014] In a fourth aspect, the present invention provides a rail crane, which is used to perform the above method or includes the above electronic device.

[0015] Beneficial effects: The obstacle recognition method provided by the present invention can effectively solve the problems of missed target detection and spatial misjudgment in low-visibility environments existing in the prior art by collecting image data from different dimensions (the front of the vehicle and the side of the lane). First, by using the first image acquisition device and the second image acquisition device, image data is acquired from the front of the vehicle and the side, respectively, to ensure all-round perception of the travel path of the rail crane, thereby overcoming the limitation of the traditional monocular vision system that cannot fully identify obstacles in complex environments. Secondly, through the fusion of multi-dimensional image data, the present invention can accurately determine the position of obstacles on the crane's travel path, reducing misjudgment and missed detection. This method is particularly suitable for harsh environments such as low light and high dust concentration. While ensuring accurate identification of obstacles, it effectively improves the safety and reliability of rail cranes and significantly improves their ability to cope with complex dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 1 is a flow chart of an obstacle identification method provided by an embodiment of the present invention; Figure 2 This is a scene example diagram of an obstacle recognition method provided by an embodiment of the present invention; Figure 3 1 is a flow chart of an obstacle identification method provided by an embodiment of the present invention; Figure 4 is a block diagram of an obstacle recognition device provided by an embodiment of the present invention; Figure 5 This is a block diagram of an electronic device used in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0018] In modern mine transportation systems, underground monorail cranes, as efficient, safe, and flexible rail-based transportation equipment, are widely used to transport materials, equipment, and personnel within mine tunnels. Their operation relies on a monorail system installed at the tunnel's roof, along which the cranes travel. However, in this complex underground environment, especially in target areas such as docking points, accurate obstacle recognition is a pressing technical challenge.

[0019] The unique characteristics of the underground environment pose significant challenges to obstacle recognition. Tunnel lighting is low and dusty, making it difficult for traditional image acquisition equipment to obtain high-quality images, which in turn affects obstacle recognition accuracy. Furthermore, the monorail crane's track is located on the tunnel's ceiling, making it even more difficult to identify obstacles ahead. Especially near stops, workers frequently come and go for loading and unloading, as well as for maintenance, increasing the potential risk of collisions.

[0020] Specifically, mine tunnels are typically 3-10 meters wide, with tracks suspended from the roof and the bottom of the crane 2-3 meters above the ground. This narrow space limits crane maneuverability and increases operational complexity. Light intensity within the tunnels is typically less than 50 lux, dust concentrations can reach 300 μg / m³, and relative humidity exceeds 70%. These factors degrade visual sensor performance and complicate obstacle detection. In loading and unloading areas, the density of people can reach 0.5 people per square meter, and equipment and materials are randomly arranged, increasing the risks and complexity of crane operations.

[0021] More specifically, in the roadway, the monorail crane can automatically stop at a stop point, such as a loading point, unloading point, or maintenance point, where staff will perform corresponding operations on the monorail crane.

[0022] Specifically, the monorail crane body is often equipped with a signal receiving device, and the stop point is often equipped with a signal sending device (for example, a beacon). Once the monorail crane travels to the communication distance of the signal sending device, it can receive the signal sent by the signal sending device to trigger the monorail crane to automatically slow down and stop.

[0023] Understandably, the communication range of these near-field communication devices is often relatively short. Specifically, the monorail crane will only receive a signal from the stop's signal transmitter when it is approaching a stop, for example, 10-15 meters from the stop, triggering it to slow down and stop. Understandably, the areas surrounding these stops are often crowded with workers. Even if the monorail crane receives the stop's signal (for example, within 10 meters of the stop) and decelerates, there is still a risk of collision with workers or equipment.

[0024] On the other hand, if a worker appears in an area beyond the stop point 10 and is on the route of the monorail crane, there is a greater risk of collision (because the monorail crane has not yet slowed down).

[0025] In one possible implementation, an image acquisition device is installed at the front of the monorail crane to capture images of the vehicle and then determine whether there are obstacles in the direction of travel. However, this approach also struggles to accurately identify obstacles in front of the vehicle and in the monorail crane's path. The main reasons are: 1. The tunnel environment is often low-light and dusty, which affects the image quality captured by the image acquisition device, and thus the accuracy of obstacle detection. Understandably, the Mie scattering effect of dust particles on visible light in the underground environment severely degrades image quality. Conventional dehazing algorithms often fail in environments with PM10 concentrations exceeding 200 μg / m³. 2. The monorail crane's track is located on the top of the tunnel, making it difficult for object detection models to determine whether workers or tools in front of the vehicle are in the crane's path (potentially causing a collision), or are simply on the sides and outside a safe distance (no collision risk), or are simply on the sides but within a safe distance (some collision risk). In other words, the track marker has been removed (due to the addition of a top), resulting in inaccurate model judgments. Understandably, traditional systems rely solely on the vehicle's monocular camera for target detection, unable to accurately restore the target's position in three-dimensional space. For example, the vehicle's camera may mistakenly identify a sidewall tool rack as a track intrusion target, ignoring the actual spatial offset, resulting in frequent false alarms. Third, the target area (the area surrounding the stop) is more likely to have multiple workers than other areas where the crane is operating (due to the need for loading and unloading cargo or vehicle maintenance, the probability of multiple workers in this area is higher). The presence of multiple workers may indicate that these workers are in the path of the rail crane, or they may not be in the path but are not at a safe distance from the crane. This increases the difficulty of identification.

[0026] In summary, during the driving process of an underground vehicle, there is a technical problem of inaccurate obstacle recognition in a target area (e.g., a stop).

[0027] like Figure 1 and Figure 2 As shown, this embodiment provides a method for identifying obstacles. The execution subject of the method may be a control device of a rail crane. The method may include: Step S12: When it is detected that the rail crane enters the target area inside the tunnel, at least first-dimensional image data and second-dimensional image data are acquired; wherein, the first-dimensional image data is image data acquired by a first image acquisition device, and the second-dimensional image data is image data acquired by a second image acquisition device; wherein, the first image acquisition device is an image acquisition device configured at the front of the rail crane for acquiring image data of the traveling direction of the rail crane, and the second image acquisition device is configured at the tunnel side of the target area for acquiring image data of the target area.

[0028] In this embodiment, the target area is the area surrounding a stop point within an underground tunnel. Specifically, the target area can be located on either side of the stop point. A wireless beacon device can be installed at the entrance or exit of the target area. The wireless beacon device emits a corresponding signal. When a corresponding module of the rail crane (e.g., a radio frequency identification module) receives the signal, it triggers the rail crane to acquire image data of the first dimension and image data of the second dimension. It will be understood that the rail crane can operate in a circular manner. Therefore, when operating from one direction to another, the entrance position at that time is the entrance position. When operating in a reverse direction, that is, when operating from another direction to one direction, the entrance position at that time is the exit position.

[0029] Understandably, the target area is prone to the presence of workers, particularly during loading and unloading operations or crane maintenance. This area is often the site of frequent interaction between workers and the crane. Compared to other areas in the underground tunnel, the target area, due to its unique functionality and high density of personnel, demands the most attention during rail crane operation. As the crane approaches its docking point, worker activity increases safety risks in this area, making obstacle detection particularly crucial.

[0030] In this embodiment, the target area may be 10 meters, 15 meters, or 30 meters in length extending in both the forward and backward directions with the docking point as the origin.

[0031] In this embodiment, the first image acquisition device can be directly positioned at the front of the rail crane, specifically, in the center of the front or slightly toward the front of the crane, to ensure that it can capture a clear image of the area in front of the crane's direction of travel. The first image acquisition device can utilize a high-resolution camera and transmit real-time captured image data to the rail crane's control device.

[0032] In this embodiment, a second image acquisition device can be deployed on the side of the tunnel within the target area, for example, on the left or right side, or both. A single second image acquisition device can be installed within the target area, or multiple second image acquisition devices can be deployed at regular intervals to cover the target area. Specifically, a second image acquisition device can be deployed at intervals of 2-3 meters to ensure continuous and comprehensive coverage of both sides of the target area and in front of it. This configuration effectively prevents blind spots and ensures that all corners of the target area and any potential obstacles are detected promptly. The second image acquisition device can be installed at a lateral viewing angle toward the target area, thereby supplementing areas not covered by the vehicle's frontal view. These image acquisition devices ensure that data can be captured from both sides of the target area and in front of it, particularly for obstacles that may exist on either side of the rail crane's path (e.g., personnel, equipment, stacked materials, etc.).

[0033] In this embodiment, the second image acquisition device also uses a high-resolution camera and also includes a corresponding communication module, which can receive signals sent by the rail crane (the rail is also equipped with a corresponding communication module, which can send the signal generated by the control device to the communication module of the second image acquisition device to trigger the second image acquisition device to capture a real-time image of the target area, and the second image acquisition device then transmits the image to the rail crane through the communication module).

[0034] In this embodiment, the first image acquisition device can be a high-resolution visible light camera, an infrared camera, or a camera with a multi-lens array. The first image acquisition device can transmit the acquired image data to the control device of the rail crane through the rail crane's own communication system.

[0035] In this embodiment, the second image acquisition device can be a high-resolution visible light camera, an infrared camera, or a camera with a multi-lens array. The acquired image data can be transmitted to the control device of the rail crane via a wireless communication module (e.g., a Wi-Fi, Bluetooth, or LTE communication module).

[0036] In this embodiment, the control device of the rail crane may be an embedded control device, an industrial PC, a PLC, a distributed control system; or an MCU, a DSP, etc.

[0037] In this embodiment, the image data of the first dimension is image data representing the target area acquired from the perspective of the vehicle head.

[0038] In this embodiment, the image data of the second dimension is image data representing the target area captured from a side perspective.

[0039] In this embodiment, the detection of the rail crane entering the target area inside the tunnel may be that the rail crane receives a signal from a beacon set at the entrance of the target area, thereby determining that the crane has entered the target area inside the tunnel.

[0040] In this embodiment, the rail crane may also determine whether it has entered the target area based on its own positioning system (eg, GPS / indoor positioning system).

[0041] In this embodiment, the crane can also determine whether it has entered the target area based on its wheel speed sensor system. That is, the crane's wheel speed sensors are used to measure the distance traveled by the crane. By recording the distance from the crane's current position to the target area in real time, the crane automatically determines that it has entered the target area when the crane's mileage counter reaches a preset distance.

[0042] In this embodiment, when it is detected that the rail crane enters the target area inside the tunnel, in addition to obtaining first-dimensional image data and second-dimensional image data, third-dimensional image data can also be obtained.

[0043] This third-dimensional image data is image data captured by a third image acquisition device. This third image acquisition device can be an image acquisition device located at the top of the tunnel in the target area. A single device or multiple devices spaced at regular intervals can be provided. The specific hardware configuration of this third image acquisition device can be the same as that of the second image acquisition device. It should be understood that this third-dimensional image data is image data of the target area captured from the perspective of the top.

[0044] Step S14: Determine whether there is an obstacle on the path of the rail crane based on at least the image data of the first dimension and the image data of the second dimension.

[0045] In this embodiment, it can be determined whether there is an obstacle on the path of the rail crane based on the first-dimensional image data and the second-dimensional image data.

[0046] Specifically, first, based on the image data of the first dimension and the image data of the second dimension, target detection is performed to obtain a first detection target and a second detection target; wherein, the first detection target is the detection target indicated in the image data of the first dimension, and the second detection target is the detection target indicated in the image data of the second dimension.

[0047] Then, the first detection target and the second detection target are unified into a path coordinate system based on the moving direction of the rail crane.

[0048] Finally, based on the coordinate distribution of each detection target in the path coordinate system, it is determined whether there is an obstacle; wherein the obstacle is a target whose projection point is located within the geometric boundary of the rail crane's travel path and whose real-time distance to the rail crane is less than a safety threshold.

[0049] In this embodiment, it is also possible to determine whether there is an obstacle on the path of the rail crane based on the first-dimensional image data, the second-dimensional image data, and the third-dimensional image data.

[0050] Specifically, first, based on the image data of the first dimension, the image data of the second dimension, and the image data of the third dimension, target detection is performed to obtain a first detection target, a second detection target, and a third detection target; wherein, the first detection target is a detection target indicated in the image data of the first dimension, the second detection target is a detection target indicated in the image data of the second dimension, and the third detection target is a detection target indicated in the image data of the third dimension.

[0051] Then, the first detection target, the second detection target, and the third detection target are unified into a path coordinate system based on the moving direction of the rail crane.

[0052] Finally, based on the coordinate distribution of each detection target in the path coordinate system, it is determined whether there is an obstacle; wherein the obstacle is a target whose projection point is located within the geometric boundary of the rail crane's travel path and whose real-time distance to the rail crane is less than a safety threshold.

[0053] The obstacle recognition method provided in this embodiment can effectively solve the problems of missed target detection and spatial misjudgment in low-visibility environments that exist in the prior art by collecting image data from different dimensions (the front of the vehicle and the side of the lane). First, by using the first image acquisition device and the second image acquisition device, image data is acquired from the front of the vehicle and the side, respectively, ensuring all-round perception of the travel path of the rail crane, thereby overcoming the limitation of traditional monocular vision systems that cannot fully identify obstacles in complex environments. Secondly, through the fusion of multi-dimensional image data, this method can accurately determine the position of obstacles on the crane's travel path, reducing misjudgment and missed detection. This method is particularly suitable for harsh environments such as low light and high dust concentration. While ensuring accurate identification of obstacles, it effectively improves the safety and reliability of rail cranes and significantly improves their ability to cope with complex dynamic environments.

[0054] In some embodiments, a wireless beacon device is provided at the entrance of the target area, and the rail crane is equipped with a radio frequency identification module that matches the wireless beacon device.

[0055] In this embodiment, the wireless beacon device is used to send a signal to the rail crane. Its primary function is to mark the entrance to the target area and transmit the signal to the rail crane via wireless communication. When the rail crane receives the beacon signal, it can confirm that it has entered the target area, thereby triggering subsequent actions.

[0056] Wireless beacons continuously transmit radio frequency signals (e.g., RF signals) containing pre-defined identification and zone information. When a rail-mounted crane approaches, the RFID module receives and decodes these signals to confirm whether the crane has entered the target zone. The RFID module is a hardware component installed on the rail-mounted crane that receives and decodes the signals emitted by the wireless beacon. Working in conjunction with the wireless beacon, the RFID module helps the crane determine whether it has entered the target zone and trigger appropriate actions.

[0057] The wireless beacon device may be an active wireless beacon, for example, a Bluetooth Low Energy (BLE) beacon, which may broadcast signals periodically using Bluetooth Low Energy technology.

[0058] The wireless beacon device may be a passive wireless beacon, for example, a UHF RFID tag.

[0059] The wireless beacon device may be an ultra-wideband (UWB) beacon.

[0060] The radio frequency identification module may be a low frequency (LF) RFID module, a high frequency (HF) RFID module, or an ultra high frequency (UHF) RFID module, etc.

[0061] like Figure 3 As shown, when the rail crane is detected to enter the target area inside the tunnel, the step of acquiring at least the image data of the first dimension and the image data of the second dimension includes: Step S122: When the rail crane travels into the communication range of the wireless beacon device, the rail crane receives the area entry signal sent by the wireless beacon device through the radio frequency identification module.

[0062] In this embodiment, the area entry signal may be a radio frequency signal transmitted by a wireless beacon device. It may include a target area identifier, a timestamp, or an expiration date. The area entry signal may be a Bluetooth Low Energy (BLE) signal or an Ultra-Wideband (UWB) signal.

[0063] Step S124: In response to the area entry signal, a synchronization start instruction is sent to the first image acquisition device and the second image acquisition device; wherein the synchronization start instruction is used to trigger the first image acquisition device and the second image acquisition device to simultaneously acquire corresponding image data.

[0064] In this embodiment, the control device of the crane can send a synchronization start instruction to the first image acquisition device in a wired or wireless manner. The control device of the crane can send a synchronization start instruction to the second image acquisition device in a wireless manner.

[0065] Specifically, the crane control device can send a synchronization start instruction to the first image acquisition device via a control line (e.g., a serial bus, CAN bus, Ethernet, or RS-485 communication protocol). The crane control device can then send the synchronization start instruction to the second image acquisition device via a wireless communication technology (e.g., Wi-Fi, Bluetooth, Zigbee, or LoRa).

[0066] In this embodiment, when the control device sends a synchronization start instruction to the first and second image acquisition devices, it can use a global clock to ensure that the two devices begin image acquisition at very close times. This time synchronization mechanism ensures synchronization between the two devices, even when using different communication methods (wired and wireless).

[0067] Step S126 : receiving the image data sent by the first image acquisition device and the second image acquisition device respectively, thereby obtaining image data of the first dimension and image data of the second dimension.

[0068] This embodiment achieves accurate detection of a rail crane entering a target area by installing a wireless beacon device at the entrance of the target area and equipping it with a radio frequency identification module that matches the beacon device. When the rail crane travels within the communication range of the beacon, the radio frequency identification module of the rail crane receives the area entry signal sent by the beacon and, in response to the signal, triggers the first image acquisition device and the second image acquisition device to start simultaneously, ensuring that when the rail crane enters the target area, image data from different perspectives can be synchronously acquired. In this way, not only is the real-time and accuracy of the image data improved, but recognition errors caused by signal delays or inconsistent images at different acquisition times are also effectively avoided. Ultimately, this synchronous acquisition mechanism can provide rail cranes with more accurate obstacle recognition, enhancing overall safety and operational efficiency.

[0069] In some embodiments, the step of determining whether there is an obstacle in the path of the rail crane based on at least the first-dimensional image data and the second-dimensional image data includes: Step S142: Perform target detection based on the image data of the first dimension and the image data of the second dimension to obtain a first detection target and a second detection target; wherein the first detection target is a detection target indicated in the image data of the first dimension, and the second detection target is a detection target indicated in the image data of the second dimension.

[0070] In this embodiment, the image data of the first dimension and the image data of the second dimension may be input into a preset target detection model to perform target detection to obtain a first detection target and a second detection target.

[0071] The target detection model may be pre-trained and deployed in a control device of the rail crane for detecting a first detection target and a second detection target.

[0072] Specifically, the target detection model may be a target detection model from the YOLO (You Only Look Once) series, or a Faster R-CNN (Region-based Convolutional Neural Networks) target detection model.

[0073] The aforementioned object inspection model can be pre-trained by collecting image data covering various environments (e.g., different lighting conditions, different people and objects), and manually annotating the location and type of obstacles. The model can then be trained using this annotated image data.

[0074] Step S144: unify the first detection target and the second detection target into a path coordinate system based on the moving direction of the rail crane.

[0075] In this embodiment, the origin of the path coordinate system can be the projection of the center point of the rail crane head on the ground, or a fixed reference point in the target area. The X-axis of the path coordinate system extends along the direction of travel of the rail crane, and the Y-axis is the direction perpendicular to the X-axis in the horizontal plane; the Z-axis of the path coordinate system is the coordinate axis perpendicular to the ground in the target area.

[0076] In this embodiment, especially when there is only one piece of image data for both the first-dimensional image data and the second-dimensional image data, that is, a plurality of detection targets are extracted from a piece of first-dimensional image data, and a plurality of detection targets are extracted from a piece of second-dimensional image data, the first detection target and the second detection target can be unified into a path coordinate system based on the travel direction of the rail crane in the following manner.

[0077] First, obtain first calibration parameters of the first image acquisition device and second calibration parameters of the second image acquisition device; wherein, the first calibration parameters include the intrinsic parameter matrix of the first image acquisition device and the extrinsic parameter matrix from the vehicle head coordinate system to the path coordinate system, and the second calibration parameters include the intrinsic parameter matrix of the second image acquisition device and the extrinsic parameter matrix from the side coordinate system to the path coordinate system.

[0078] In this embodiment, the origin of the vehicle-mounted coordinate system can be the optical center point of the first image acquisition device. The X-axis is along the direction of travel of the rail crane (consistent with the direction of track extension). The Y-axis is horizontally perpendicular to the track (positive in the right direction). The Z-axis is perpendicular to the ground and upward (positive in the top direction). The vehicle-mounted coordinate system represents the direct observation data of the vehicle-mounted camera on obstacles ahead and is used to calculate the relative position of the target.

[0079] In this embodiment, the origin of the sidewall coordinate system can be the optical center point of the second image acquisition device. The X-axis extends along the roadway (parallel to the track). The Y-axis is perpendicular to the roadway sidewall surface (positive direction toward the roadway center). The Z-axis is perpendicular to the ground and upward (positive direction toward the top). The sidewall coordinate system provides observation data from a side perspective, compensating for the blind spots of the vehicle's front camera and is particularly suitable for detecting obstacles near roadway sidewalls.

[0080] Specifically, for the first calibration parameter: The intrinsic parameter matrix can include parameters such as the camera's focal length, principal point, and distortion coefficients. The intrinsic parameter matrix is ​​a fundamental tool for mapping the two-dimensional pixel coordinates in an image to the camera's coordinate system. For the front camera (i.e., the first image acquisition device), its intrinsic parameter matrix describes how each pixel in the image is associated with a three-dimensional point in the camera's coordinate system.

[0081] In the formula, Expressed as an internal parameter matrix, Expressed as focal length (along the x-axis and y-axis), are the coordinates of the principal point of the image (the center of the image).

[0082] The extrinsic matrix describes the relationship between the vehicle coordinate system and the path coordinate system, namely the rotation matrix and displacement vector from the vehicle coordinate system to the path coordinate system. The extrinsic matrix is ​​used to transform the 3D points in the vehicle coordinate system into the path coordinate system for subsequent coordinate transformations.

[0083] In the formula, is the rotation matrix, the rotation relationship between the vehicle coordinate system and the path coordinate system, is the translation vector, which represents the displacement relationship between the origin of the vehicle coordinate system and the origin of the path coordinate system.

[0084] Specifically, for the second calibration parameter: The intrinsic parameter matrix may be similar to that of the front camera, and is used to describe how the image captured by the side camera is converted into three-dimensional coordinates in the camera coordinate system.

[0085] The extrinsic parameter matrix in the second calibration parameter is used to describe the relationship between the second image acquisition device (side coordinate system) and the path coordinate system. It is used to map the three-dimensional point coordinates captured by the side camera to the path coordinate system of the rail crane.

[0086] Then, based on the first calibration parameters, the two-dimensional pixel coordinates in the image data of the first dimension are converted into three-dimensional coordinates in a path coordinate system.

[0087] Step 1: Use the intrinsic parameter matrix of the front camera to convert the two-dimensional pixel coordinates in the image into three-dimensional coordinates in the camera coordinate system.

[0088] Step 2: According to the external parameter matrix of the front camera, convert the three-dimensional coordinates in the camera coordinate system into the path coordinate system.

[0089] Through this transformation, the three-dimensional position of the target in the path coordinate system is obtained.

[0090] Next, based on the second calibration parameters, the two-dimensional pixel coordinates in the image data of the second dimension are converted into three-dimensional coordinates in a path coordinate system.

[0091] Similarly, the intrinsic parameter matrix of the side camera is used to map the 2D pixel coordinates in the image to the 3D coordinates in the camera coordinate system. The extrinsic parameter matrix of the side camera is then used to transform the 3D coordinates in the side camera coordinate system into the path coordinate system.

[0092] Finally, a preset data fusion method is used to fuse the three-dimensional coordinates obtained from the first coordinate transformation and the second coordinate transformation to generate the final coordinates of each target in the path coordinate system.

[0093] After completing the above coordinate transformation, all target coordinates obtained from the first image acquisition device and the second image acquisition device are converted into three-dimensional coordinates in the path coordinate system. Since the two image data have different sources and angles, these coordinates need to be fused.

[0094] The preset data fusion method can be a weighted average method or a least squares method.

[0095] Step S146: Determine whether there is an obstacle based on the coordinate distribution of each detection target in the path coordinate system; wherein the obstacle is a target whose projection point is within the geometric boundary of the rail crane's travel path and whose real-time distance from the rail crane is less than a safety threshold.

[0096] This embodiment combines geometric boundary conditions and safety threshold conditions to accurately determine whether there are obstacles on the path of the rail crane, thereby achieving a more accurate collision risk assessment.

[0097] Specifically, the system first identifies a first and second target using an object detection method based on the first and second dimensional image data. These two targets are then converted to a unified path coordinate system, ensuring their spatial locations are within the actual coordinate framework of the crane's travel path. Finally, the system performs a judgment based on the coordinate distribution of these targets. Only when an target satisfies both of these conditions is it considered an obstacle. The geometric boundary condition ensures that the detected target is within the geometric boundaries of the crane's travel path. In other words, the obstacle must be within the crane's actual physical path and not in a nearby safety zone. This condition effectively excludes objects or locations unrelated to the crane's travel path, ensuring that the system identifies obstacles that could potentially collide with the crane. The safety threshold condition ensures that the obstacle is not only within the crane's travel path but also within a predefined safety threshold. This safety threshold is calculated in real time based on the crane's current speed, braking distance, and dynamic environmental factors. This ensures that any obstacles identified are close enough to the crane to pose a real collision risk. The combined assessment of these two conditions is crucial. An obstacle is considered a potential collision risk only when it is both in the hazardous space (i.e., on the driving path) and too close to the crane (i.e., within the collision warning range). This approach effectively avoids false alarms and unnecessary reactions (such as premature deceleration or stopping). It also ensures that the crane can take timely action when a real collision risk arises, thereby improving the safety and operational efficiency of rail cranes in complex environments.

[0098] In some embodiments, the X-axis of the path coordinate system extends along the direction of travel of the rail crane, and the Y-axis is a direction perpendicular to the X-axis in the horizontal plane; The step of determining whether there is an obstacle based on the coordinate distribution of each detection target in the path coordinate system includes: Step S1462: Determine whether there is a target that meets the first judgment condition; wherein, the first judgment condition is that the absolute value of the target's coordinate in the Y-axis direction is less than or equal to the sum of half the track projection width and a preset lateral safety margin.

[0099] In this embodiment, the track projection width can be pre-stored in the control device. For example, during the initialization phase, the track top structure is scanned by a laser radar to generate a projection model of the track centerline on the ground.

[0100] In this embodiment, the preset lateral safety margin is used to compensate for vehicle body swing and measurement errors.

[0101] In this embodiment, the coordinate distribution of each detection target in the path coordinate system can be traversed to calculate the absolute value of each target's coordinate in the Y-axis direction. The calculated absolute value is then compared with the sum of half the track projection width and a preset lateral safety margin to obtain a judgment result.

[0102] In an embodiment, the absolute value of the coordinate of the target in the Y-axis direction represents the absolute value from the target to the origin in the Y-axis direction, and the origin may be a fixed reference point on the bottom surface of the projection of the track center.

[0103] Step S1464, determine whether there is a target that meets the second judgment condition; wherein, the second judgment condition indicates that the coordinate value of the target in the X-axis direction is less than or equal to the safety threshold; wherein, the safety threshold is represented by a safety distance threshold dynamically calculated based on the current speed of the rail crane and the braking parameters.

[0104] In an embodiment, the second judgment condition is used to determine whether the target is in a longitudinal collision risk area. The expression of the safety threshold is: In the formula, Expressed as a safety threshold, It represents the current real-time speed of the rail crane, which can be obtained through the encoder. Expressed as the total system response time, it can be a fixed value, for example, 1.2-1.5 seconds. Expressed as the maximum braking deceleration (braking parameter), it can be calculated based on the track friction coefficient. It is expressed as the sensor error compensation amount and can be determined through calibration experiments.

[0105] In an embodiment, environmental parameters may be collected in real time to update the braking parameters, which may be updated using the following expression: In the formula, It is expressed as the maximum braking deceleration when the track is dry, and and It is the environmental parameter collected in real time. The higher it is, the more dust there is and the lower the maximum braking deceleration will be.

[0106] Step S1466: Determine the target that meets both the first judgment condition and the second judgment condition as an obstacle.

[0107] In this implementation, by establishing a dynamic path coordinate system strongly coupled with the motion characteristics of the rail crane and combining it with a dual-dimensional joint judgment mechanism, a breakthrough in both accuracy and adaptability of obstacle detection in complex underground environments is achieved. Dynamic adjustment of the Y-axis threshold based on the projected track width (for example, the safety margin expands to ±1.8m for a 3m track width) effectively eliminates boundary misjudgments caused by tunnel deformation and vehicle sway in traditional solutions. Furthermore, by integrating an X-axis threshold calculation model with vehicle speed and braking performance parameters, the safety distance adaptively adjusts to the operating conditions, avoiding transport efficiency losses caused by excessive deceleration. Furthermore, by requiring both lateral intrusion and longitudinal threat conditions to be met, this approach avoids false triggering of sidewall equipment (e.g., tool racks) caused by a single distance threshold and prevents missed detection of distant track targets, ultimately improving overall detection accuracy in densely populated loading and unloading areas.

[0108] In some embodiments, the step of unifying the first detection target and the second detection target into a path coordinate system based on the travel direction of the rail crane includes: Step S1442: Obtain first calibration parameters of the first image acquisition device and second calibration parameters of the second image acquisition device; wherein the first calibration parameters include the intrinsic parameter matrix of the first image acquisition device and the extrinsic parameter matrix from the vehicle head coordinate system to the path coordinate system, and the second calibration parameters include the intrinsic parameter matrix of the second image acquisition device and the extrinsic parameter matrix from the side coordinate system to the path coordinate system.

[0109] Step S1444: Based on the first calibration parameters, convert the two-dimensional pixel coordinates in the image data of the first dimension into three-dimensional coordinates in a path coordinate system.

[0110] Step S1446: Based on the second calibration parameters, convert the two-dimensional pixel coordinates in the image data of the second dimension into three-dimensional coordinates in the path coordinate system.

[0111] Step S1448: Using a preset data fusion method, perform data fusion on the three-dimensional coordinates obtained by the first coordinate transformation and the second coordinate transformation to generate the final coordinates of each target in the path coordinate system.

[0112] This embodiment uses precise calibration parameters and coordinate transformation methods to uniformly transform detected targets in image data from different viewpoints (front and side) into the rail crane's path coordinate system. First, calibration parameters, including intrinsic and extrinsic matrices, are obtained for the first and second image acquisition devices to ensure accurate mapping between image coordinates and three-dimensional spatial coordinates. Then, based on these calibration parameters, the two-dimensional pixel coordinates in the first and second dimensional image data are converted to three-dimensional coordinates in the path coordinate system, eliminating errors introduced by different viewpoints. Finally, using a pre-defined data fusion method, the coordinate data from the front and side views are combined to generate the final coordinates of the target in the path coordinate system. This process significantly improves target detection accuracy, ensuring that the rail crane can accurately determine the location of obstacles along its travel path, thereby enhancing safety, reducing collision risks, and effectively enhancing the intelligence and automation of crane operations.

[0113] In this embodiment, the vehicle front coordinate system, the side coordinate system, and the path coordinate system may all be pre-calibrated.

[0114] In one possible implementation, when the origin of the path coordinate system is a fixed reference point in the target area, pre-calibration can be performed using the following scheme. Specifically, a total station can be used to measure the center coordinates of the stop point to establish a global coordinate system, and a laser transmitter can be installed on the top of the tunnel to define the track direction vector. For the vehicle head coordinate system, the rail crane can be accurately parked at the origin, with the vehicle head aligned in the X-axis direction. Calibration plates can be placed 5m and 10m in front of the vehicle head to ensure that the plate surface is parallel to the XY plane. The intrinsic parameter matrix and the extrinsic parameter matrix are then calculated using Zhang Zhengyou's calibration method. For the side coordinate system, multiple control points with known coordinates (for example, LED light spots) can be arranged in the global coordinate system. The control points are observed by the side camera to establish a perspective projection model. The perspective projection model is then solved to obtain the extrinsic parameter matrix.

[0115] In some embodiments, the step of performing target detection based on the image data of the first dimension and the image data of the second dimension to obtain a first detected target and a second detected target includes: Step S1422: Perform preset image preprocessing on the image data of the first dimension and the image data of the second dimension to obtain preprocessed image data of the first dimension and preprocessed image data of the second dimension.

[0116] In this embodiment, step S1422 may include: Step S14222: Determine the target preprocessing method based on the environmental parameters.

[0117] The environmental parameter may be dust concentration. The dust concentration may be acquired in real time or in a previous step. For example, in step S12, the environmental parameter is acquired simultaneously with the acquisition of the image data. Specifically, the dust concentration may be acquired using a laser scattering sensor pre-installed on the crane.

[0118] In this embodiment, if the dust concentration is greater than the dust concentration threshold, the spectrum fusion preprocessing method is determined as the target preprocessing method.

[0119] In this embodiment, if the dust concentration is not greater than the dust concentration threshold, the target preprocessing method may be denoising (Gaussian filtering, mean filtering, etc.), or adaptive contrast enhancement, etc.

[0120] Step S14224: Use the target preprocessing method to perform corresponding preprocessing on the image data of the first dimension and the image data of the second dimension.

[0121] In this embodiment, the multispectral fusion preprocessing method includes: First, feature extraction is performed on the visible light image and near-infrared image included in the first-dimensional image data and the second-dimensional image data (it is understood that the image data in both dimensions can include visible light images and near-infrared images). It is understood that near-infrared images capture infrared information in the invisible spectrum and can penetrate some atmospheric interference (for example, haze, dust, etc.), performing well in low-visibility environments.

[0122] Specifically, we can use layer 4 of the VGG16 network to extract deep features from visible light images. VGG16 is a convolutional neural network used for image processing, capable of extracting multi-level feature information from images. Layer 4 is a layer that extracts high-level features, capable of capturing complex patterns, textures, and other information in an image.

[0123] The block 3 layer of the ResNet (residual network) model can be used to extract mid-level features from near-infrared images. ResNet models are particularly adept at processing deep image features and effectively extracting detailed information. Here, block 3 is selected to extract mid-level features in order to exploit the texture and structural information in near-infrared images.

[0124] Then, the fusion weight is determined based on the dust concentration.

[0125] Specifically, the fusion weight may be 1-(dust concentration / 1000).

[0126] Then, the features of the visible light image and the features of the near-infrared image are fused through the fusion weights to obtain a fused feature map.

[0127] Specifically, a fusion weight can be assigned to the features of the visible light image, and (1-fusion weight) can be assigned to the features of the near-infrared image.

[0128] Finally, the fused feature map is input into a generator network (such as U-Net) for image reconstruction to obtain reconstructed image data of the first dimension and image data of the second dimension.

[0129] Step S1424 : Target detection is performed on the pre-processed image data of the first dimension and the pre-processed image data of the second dimension, respectively, to obtain a first detected target and a second detected target.

[0130] In this embodiment, a target detection model (e.g., YOLO, Faster R-CNN, SSD, etc.) may be used to perform target detection on the preprocessed first-dimensional image data and the preprocessed second-dimensional image data to obtain a first detected target and a second detected target. This embodiment significantly improves the accuracy and robustness of target detection by performing preprocessing and target detection on image data of the first dimension and the second dimension. First, through the preset image preprocessing steps, the noise, dust and interference in low-light environments in the image are effectively removed, the visibility of the target is enhanced, and clearer and more reliable input data is provided for subsequent target detection. Secondly, target detection is performed on the preprocessed image data, which can accurately identify obstacles in the image and determine their position and category. This process not only improves the accuracy of target detection and avoids false alarms and missed alarms, but also enhances the system's adaptability in complex environments, ensuring that rail cranes can efficiently and safely avoid obstacles in actual work and reduce potential collision risks. Overall, this embodiment significantly improves the safety and reliability of the crane during driving through the combination of image preprocessing and target detection.

[0131] In some embodiments, the step of performing preset image preprocessing on the image data of the first dimension and the image data of the second dimension to obtain preprocessed image data of the first dimension and preprocessed image data of the second dimension includes: A preset image defogging algorithm is executed on the image data of the first dimension and the image data of the second dimension to obtain the image data of the first dimension and the image data of the second dimension after the image defogging.

[0132] In this embodiment, the preset image defogging algorithm may be a dark primary color prior defogging algorithm.

[0133] In this embodiment, the preset image defogging algorithm can be the Retinex defogging algorithm In this embodiment, the preset image defogging algorithm may be an adaptive histogram equalization algorithm.

[0134] According to an embodiment of the present invention, an obstacle recognition device is provided. Figure 4 , the device comprises: an acquisition module configured to acquire at least first-dimensional image data and second-dimensional image data upon detecting that the rail crane enters a target area within a laneway; wherein the first-dimensional image data is image data acquired by a first image acquisition device, and the second-dimensional image data is image data acquired by a second image acquisition device; wherein the first image acquisition device is an image acquisition device disposed at the front of the rail crane for acquiring image data in the direction of travel of the rail crane, and the second image acquisition device is disposed at a laneway sidewall in the target area for acquiring image data of the target area; A determination module is configured to determine whether there is an obstacle on a path of the rail crane based at least on the image data of the first dimension and the image data of the second dimension.

[0135] According to an embodiment of the present invention, an electronic device is provided. Figure 5 The electronic device in this embodiment may include one or more of the following components: a processor, a network interface, a memory, a non-volatile memory, and one or more applications, wherein the one or more applications may be stored in the non-volatile memory and configured to be executed by one or more processors, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0136] According to an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer executes the method described in any one of the above embodiments.

[0137] According to an embodiment of the present invention, a computer program product comprising instructions is further provided. When the instructions are executed by a computer, the computer is enabled to perform a method in any one of the above embodiments.

[0138] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0139] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0140] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0141] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0142] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying an obstacle, characterized in that: include: When it is detected that the rail crane enters a target area inside the tunnel, at least first-dimensional image data and second-dimensional image data are acquired; wherein the first-dimensional image data is image data acquired by a first image acquisition device, and the second-dimensional image data is image data acquired by a second image acquisition device; wherein the first image acquisition device is an image acquisition device configured at the front of the rail crane for acquiring image data in the direction of travel of the rail crane, and the second image acquisition device is configured at the side of the tunnel in the target area for acquiring image data of the target area; Based on at least the first-dimensional image data and the second-dimensional image data, it is determined whether there is an obstacle on the path of the rail crane.

2. The method according to claim 1, characterized in that A wireless beacon device is provided at the entrance of the target area, and the rail crane is equipped with a radio frequency identification module that matches the wireless beacon device; The step of acquiring at least first-dimensional image data and second-dimensional image data when detecting that the rail crane enters the target area inside the tunnel comprises: When the rail crane travels into the communication range of the wireless beacon device, the rail crane receives the area entry signal sent by the wireless beacon device through the radio frequency identification module; In response to the area entry signal, sending a synchronization start instruction to the first image acquisition device and the second image acquisition device; wherein the synchronization start instruction is used to trigger the first image acquisition device and the second image acquisition device to simultaneously acquire corresponding image data; Image data sent by the first image acquisition device and the second image acquisition device are received respectively, thereby obtaining image data of the first dimension and image data of the second dimension.

3. The method according to claim 1, characterized in that The step of determining whether there is an obstacle on the path of the rail crane based on at least the first-dimensional image data and the second-dimensional image data includes: Performing target detection based on the image data of the first dimension and the image data of the second dimension to obtain a first detected target and a second detected target; wherein the first detected target is a detected target indicated in the image data of the first dimension, and the second detected target is a detected target indicated in the image data of the second dimension; Unifying the first detection target and the second detection target into a path coordinate system based on the traveling direction of the rail crane; Based on the coordinate distribution of each detection target in the path coordinate system, it is determined whether there is an obstacle; wherein the obstacle is a target whose projection point is located within the geometric boundary of the rail crane's travel path and whose real-time distance from the rail crane is less than a safety threshold.

4. The method according to claim 3, characterized in that The X-axis of the path coordinate system extends along the travel direction of the rail crane, and the Y-axis is a direction perpendicular to the X-axis in the horizontal plane; The step of determining whether there is an obstacle based on the coordinate distribution of each detection target in the path coordinate system includes: Determine whether a target satisfies a first judgment condition; wherein the first judgment condition is that the absolute value of the target's coordinate in the Y-axis direction is less than or equal to the sum of half the track projection width and a preset lateral safety margin; Determine whether a target satisfies a second judgment condition; wherein the second judgment condition is that the coordinate value of the target in the X-axis direction is less than or equal to a safety threshold; wherein the safety threshold is a safety distance threshold dynamically calculated based on the current speed and braking parameters of the rail crane; An object that satisfies both the first judgment condition and the second judgment condition is judged as an obstacle.

5. The method according to claim 4, characterized in that The step of unifying the first detection target and the second detection target into a path coordinate system based on the traveling direction of the rail crane comprises: Obtaining first calibration parameters of the first image acquisition device and second calibration parameters of the second image acquisition device; wherein the first calibration parameters include an intrinsic parameter matrix of the first image acquisition device and an extrinsic parameter matrix from the vehicle head coordinate system to the path coordinate system, and the second calibration parameters include an intrinsic parameter matrix of the second image acquisition device and an extrinsic parameter matrix from the side coordinate system to the path coordinate system; Based on the first calibration parameter, converting the two-dimensional pixel coordinates in the image data of the first dimension into three-dimensional coordinates in a path coordinate system; Based on the second calibration parameter, converting the two-dimensional pixel coordinates in the image data of the second dimension into three-dimensional coordinates in a path coordinate system; A preset data fusion method is used to fuse the three-dimensional coordinates obtained by the first coordinate transformation and the second coordinate transformation to generate the final coordinates of each target in the path coordinate system.

6. The method according to claim 3, characterized in that The step of performing target detection based on the image data of the first dimension and the image data of the second dimension to obtain a first detected target and a second detected target includes: Performing preset image preprocessing on the image data of the first dimension and the image data of the second dimension to obtain preprocessed image data of the first dimension and preprocessed image data of the second dimension; Target detection is performed on the preprocessed image data of the first dimension and the preprocessed image data of the second dimension, respectively, to obtain a first detected target and a second detected target.

7. The method according to claim 6, characterized in that The step of performing preset image preprocessing on the image data of the first dimension and the image data of the second dimension to obtain preprocessed image data of the first dimension and preprocessed image data of the second dimension includes: A preset image defogging algorithm is executed on the image data of the first dimension and the image data of the second dimension to obtain the image data of the first dimension and the image data of the second dimension after the image defogging.

8. An obstacle recognition device, characterized in that: include: an acquisition module configured to acquire at least first-dimensional image data and second-dimensional image data upon detecting that the rail crane enters a target area within a laneway; wherein the first-dimensional image data is image data acquired by a first image acquisition device, and the second-dimensional image data is image data acquired by a second image acquisition device; wherein the first image acquisition device is an image acquisition device disposed at the front of the rail crane for acquiring image data in the direction of travel of the rail crane, and the second image acquisition device is disposed at a laneway sidewall in the target area for acquiring image data of the target area; A determination module is configured to determine whether there is an obstacle on a path of the rail crane based at least on the image data of the first dimension and the image data of the second dimension.

9. An electronic device, characterized in that: include: a memory, and one or more processors communicatively coupled to the memory; Instructions executable by the one or more processors are stored in the memory. The instructions are executed by the one or more processors to enable the one or more processors to implement the method according to any one of claims 1 to 7.

10. A rail crane, characterized in that: The rail crane is used to perform the method according to any one of claims 1 to 7 or includes the electronic equipment according to claim 9.

Images