Locomotive dangerous area obstacle detection method, device and equipment and storage medium

Abstract

The invention provides a locomotive dangerous area obstacle detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-detected image of a locomotive and locomotive path information, and dividing a rail driving area from the to-be-detected image based on the locomotive path information, the method comprises the steps of dividing a potential dangerous area from a to-be-detected image according to a preset steel rail distance, performing obstacle detection on the to-be-detected image based on a pre-trained obstacle detection model, obtaining an area where an obstacle is located in the to-be-detected image according to an obstacle detection result, and performing obstacle early warning according to the area where the obstacle is located. Through the method, the technical problem of inaccurate obstacle detection of the unmanned aerial vehicle in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of image recognition and repair technology, and in particular to a method, device, electronic equipment and storage medium for detecting obstacles in dangerous areas of a locomotive. Background Art

[0002] At present, deep learning technology has been widely used to identify obstacle locations in response to the scale uncertainty problem of monocular cameras. However, when this technology is applied to rail transportation scenarios, the lack of multimodal datasets becomes a significant problem in the specific scenario of rail transportation, because deep learning methods rely heavily on large and high-quality datasets to improve recognition accuracy. The scarcity of this data may cause the model to have false positives or false negatives when identifying obstacles, which in turn poses a threat to the safety of autonomous driving.

[0003] In addition, in order to make up for the shortcomings of monocular cameras in terms of scale uncertainty, related technologies combine lidar data or use binocular cameras. However, in harsh environments, lidar may be interfered by noise, resulting in large errors in the camera's recognition results. In rail transportation scenarios, since locomotives often need to be connected to heavy objects, the braking distance is relatively long. This greatly limits the detection distance of binocular cameras, making it difficult to meet the braking distance requirements of drone vehicles. Summary of the invention

[0004] In view of the shortcomings of the above-mentioned related technologies, the present application provides a method, device, electronic device and storage medium for detecting obstacles in a dangerous area of ​​a locomotive to solve the technical problem of inaccurate obstacle detection of unmanned vehicles in the related technologies.

[0005] The present application provides a method for detecting obstacles in dangerous areas of a locomotive, and the method comprises: obtaining an image to be detected of a locomotive and locomotive path information; dividing a rail running area from the image to be detected based on the locomotive path information, and dividing a potential danger area from the image to be detected according to a preset rail spacing; performing obstacle detection on the image to be detected based on a pre-trained obstacle detection model, obtaining an area where the obstacle is located in the image to be detected according to the obstacle detection result, and issuing an obstacle warning according to the area.

[0006] In one embodiment of the present application, a coordinate system is established in the image to be detected, and multiple track coordinate points in the rail travel area are extracted; the image to be detected is transversely segmented, and the track slopes of the first track and the second track in each transverse segment are fitted based on the track coordinate points, and the image to be detected includes the first track and the second track; if the track slope difference between the first track and the second track in a transverse segment is less than a preset difference threshold, the image pixel interval between the first track and the second track in the transverse segment is obtained; the scale information of the transverse segment is obtained according to the preset rail spacing and the image pixel interval between the first track and the second track; based on the scale information of each transverse segment and the size information of the locomotive, a potential danger area is divided in the image to be detected.

[0007] In one embodiment of the present application, performing obstacle warning according to the area in which the locomotive is located includes: if the area in which the locomotive is located is the potential danger area or the rail travel area, the locomotive performs obstacle warning and executes an emergency avoidance strategy.

[0008] In one embodiment of the present application, obtaining the area where the obstacle is located in the image to be detected according to the obstacle detection result includes: obtaining the pixel position of the obstacle in the image to be detected and the horizontal segment corresponding to the pixel position according to the obstacle detection result; obtaining the actual position of the obstacle relative to the locomotive based on the scale information of the horizontal segment corresponding to the pixel position and the pixel position of the obstacle. According to the actual position of the obstacle relative to the locomotive, auxiliary driving parameters are provided for the emergency avoidance strategy.

[0009] In one embodiment of the present application, dividing the rail running area from the image to be detected based on the locomotive path information includes: inputting the image to be detected into a pre-trained semantic segmentation model for semantic segmentation to extract rail features in the image to be detected; matching the rail features with the locomotive path information, and if the match is successful, dividing the rail running area from the image to be detected based on the rail features.

[0010] In one embodiment of the present application, before performing obstacle detection on the image to be detected based on a pre-trained obstacle detection model, the process further includes: acquiring historical locomotive driving images; labeling obstacles in the historical locomotive driving images based on obstacle labels to obtain an obstacle training set, wherein the obstacle labels at least include pedestrian labels, large obstacle labels, and small obstacle labels; and inputting the obstacle training set into a preset target detection for training to obtain an obstacle detection model.

[0011] In one embodiment of the present application, after obstacle detection is performed on the image to be detected based on a preset target detection model, it also includes: obtaining the obstacle type of the obstacle and the area where the obstacle is located in the image to be detected according to the obstacle detection result; if the area where the obstacle is located is the rail running area, a first-level severe alarm is issued; if the obstacle type of the obstacle is a pedestrian, and the area where the obstacle is located is the potential danger area, a first-level severe alarm is issued; if the obstacle type of the obstacle is not a pedestrian, and the area where the obstacle is located is the potential danger area, a second-level moderate alarm is issued.

[0012] An embodiment of the present application also provides a locomotive dangerous area obstacle detection device, which includes: an information input module, used to obtain a locomotive image to be detected and locomotive path information; an area division module, used to divide the rail running area from the image to be detected based on the locomotive path information, and divide the potential danger area from the image to be detected according to a preset rail spacing; a detection and warning module, used to perform obstacle detection on the image to be detected based on a pre-trained obstacle detection model, obtain the area where the obstacle is located in the image to be detected according to the obstacle detection result, and issue an obstacle warning based on the area.

[0013] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the locomotive dangerous area obstacle detection method as described in any of the above embodiments.

[0014] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is enabled to execute the method for detecting obstacles in a dangerous area of ​​a locomotive as described in any one of the above embodiments.

[0015] Beneficial effects of the present application: The embodiments of the present application provide a method, device, electronic device and storage medium for detecting obstacles in dangerous areas of a locomotive. The method comprises obtaining an image to be detected of a locomotive and locomotive path information, dividing a rail running area from the image to be detected based on the locomotive path information, dividing a potential danger area from the image to be detected according to a preset rail spacing, performing obstacle detection on the image to be detected based on a pre-trained obstacle detection model, obtaining an area where the obstacle is located in the image to be detected according to the obstacle detection result, and performing obstacle warning according to the area where the obstacle is located. The method estimates the area where the obstacle is located by the parallel characteristics of the rails, and divides the potential danger area according to the rail spacing, thereby achieving accurate graded warning of obstacles and improving the driving safety of the locomotive.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram of an implementation environment of a locomotive dangerous area obstacle detection method shown in an exemplary embodiment of the present application;

[0018] Figure 2 is a flow chart of a method for detecting obstacles in a dangerous area of ​​a locomotive, shown in an exemplary embodiment of the present application;

[0019] Figure 3 is a schematic diagram of a straight track shown in an exemplary embodiment of the present application;

[0020] Figure 4 is a flow chart of a method for detecting pedestrians in a dangerous area of ​​a locomotive, shown in an exemplary embodiment of the present application;

[0021] Figure 5 is a block diagram of a locomotive dangerous area obstacle detection device shown in an exemplary embodiment of the present application;

[0022] Figure 6 It is a structural schematic diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0025] It should be noted that in this application, "first", "second", etc. are only used to distinguish similar objects, and are not used to limit the order or precedence of similar objects. The variations of "including", "having", etc. described above indicate that the scope covered by the subject of the word is not exclusive except for the examples shown by the word.

[0026] It is understood that the various numbers, step numbers, and other reference numerals recorded in this application are distinguished for the convenience of description and are not intended to limit the scope of this application. The size of the reference numerals in this application does not mean the order of execution. The execution order of each process should be determined by its function and internal logic.

[0027] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0028] The embodiments of the present application respectively propose a locomotive dangerous area obstacle detection method, a locomotive dangerous area obstacle detection device, an electronic device, a computer-readable storage medium and a computer program product, which will be described in detail below.

[0029] See also Figure 1 , Figure 1 It is a schematic diagram of an implementation environment of a locomotive dangerous area obstacle detection method shown in an exemplary embodiment of the present application.

[0030] like Figure 1 As shown, the implementation environment may include an image acquisition device 101 and an unmanned aerial vehicle 102, wherein the image acquisition device 101 may be a vehicle-mounted camera installed on the unmanned aerial vehicle 102, and the unmanned aerial vehicle 102 is loaded with an obstacle detection model and a rail semantic segmentation model for acquiring the image to be detected acquired by the image acquisition device 101, and performing obstacle warning based on the image to be detected.

[0031] See also Figure 2 , Figure 2is a flow chart of a method for detecting obstacles in a dangerous area of ​​a locomotive, as shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is as follows. The method may also be applicable to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0032] like Figure 2 As shown, in an exemplary embodiment, the method for detecting obstacles in a dangerous area of ​​a locomotive includes at least steps S210 to S230, which are described in detail as follows:

[0033] Step S210, obtaining the image of the locomotive to be detected and the locomotive path information.

[0034] In one embodiment of the present application, a front real-time image of the unmanned locomotive during its travel is obtained from an onboard camera of the unmanned locomotive, and the front real-time image is used as an image to be detected. Locomotive path information is obtained from a locomotive dispatching center, and the locomotive path information includes the planned travel path of the locomotive. Based on the locomotive path information, it can be obtained whether the locomotive is currently on a straight road, a curve, or a switch.

[0035] Step S220: dividing the rail travel area from the image to be detected based on the path information of the locomotive, and dividing the potential danger area from the image to be detected according to the preset rail spacing.

[0036] In one embodiment of the present application, dividing the rail travel area from the image to be detected based on the locomotive path information includes: inputting the image to be detected into a pre-trained semantic segmentation model for semantic segmentation to extract rail features in the image to be detected; matching the rail features with the locomotive path information, and if the match is successful, dividing the rail travel area from the image to be detected based on the rail features.

[0037] In one embodiment of the present application, the rail feature includes a first rail and a second rail, and the rail travel area is an area between the first rail and the second rail ( Figure 3 In the present application, matching the rail features with the locomotive path information includes, according to the rail features, determining whether the rail in the current image to be detected is a straight track, a curve or a switch. According to the locomotive path information, the rail on which the current locomotive is traveling is a straight track, a curve or a switch. If the rail in the current image to be detected matches the rail on which the current locomotive is traveling successfully, then the rail travel area is divided from the image to be detected based on the rail features. Dividing the rail travel area from the image to be detected based on the rail features includes taking the area between the first track and the second track as the rail travel area.

[0038] In one embodiment of the present application, dividing a potential danger area from an image to be detected according to a preset rail scale includes: establishing a coordinate system in the image to be detected, and extracting multiple track coordinate points in the rail running area; transversely segmenting the image to be detected, and fitting the track slopes of the first track and the second track in each transverse segment based on the track coordinate points, and the image to be detected includes the first track and the second track; if the difference in track slope between the first track and the second track in a transverse segment is less than a preset difference threshold, then obtaining the image pixel interval between the first track and the second track in a transverse segment; obtaining the scale information of a transverse segment according to the preset rail spacing and the image pixel interval between the first track and the second track; dividing the potential danger area in the image to be detected based on the scale information of each transverse segment and the size information of the locomotive.

[0039] In one embodiment of the present application, establishing a coordinate system in the image to be detected includes setting the origin at the upper left corner of the image, the x-axis is parallel to the horizontal direction of the image, and the y-axis is parallel to the vertical direction of the image. In this coordinate system, each pixel position of the image can be represented by an (x, y) coordinate.

[0040] In one embodiment of the present application, before extracting multiple track coordinate points in the rail travel area, the image to be detected is preprocessed, such as denoising, contrast enhancement, etc. The processed image pixels are traversed, the rail edge in the rail travel area is detected by an edge detection algorithm, and the coordinate points on the rail are extracted based on the rail edge. In this embodiment, N track coordinate points can be extracted from the first track and the second track in each horizontal segment.

[0041] In one embodiment of the present application, horizontal segmentation of the image to be detected includes determining the number of horizontal segments according to the width of the image to be detected and the characteristics of the track. If there are too many horizontal segments, it may lead to an increase in the amount of calculation, and if there are too few horizontal segments, it may not be able to accurately reflect the changes in the track. The image to be detected is horizontally segmented according to a preset number of segments to obtain multiple horizontally segmented sub-areas.

[0042] In one embodiment of the present application, in each transverse segmented sub-area, a curve fitting algorithm (e.g., least squares polynomial fitting) is used to fit the track coordinate points on the first track and the second track, and the track slope of the first track and the track slope of the second track in each transverse segmented sub-area are obtained. The track slope of the first track in the same transverse segmented sub-area is compared with the track slope of the second track. If the track slopes are similar (in this embodiment, the difference in the track slopes between the first track and the second track is less than a preset difference threshold), it means that the first track and the second track in the transverse segmented sub-area are parallel. If the first track and the second track in the transverse segmented sub-area are parallel, the image pixel spacing between the first track and the second track (i.e., the drivable area) in the transverse segmented sub-area is obtained. Because the railway track is usually composed of two parallel rails (i.e., the first track and the second track), the track spacing in different track transportation scenarios is a fixed value, and the track spacing-related dimensional parameters of the first track and the second track in three-dimensional space (preset rail spacing) will not change with weather and camera field of view, so the scale information of each transverse segmented sub-area can be obtained according to the preset rail spacing and the image pixel spacing of each transverse segmented sub-area.

[0043] In one embodiment of the present application, the size of the unmanned vehicle and its attached cargo often exceeds the drivable area, and it is necessary to divide the potential danger area of ​​the locomotive on the basis of the above. The scale uncertainty problem of the monocular camera leads to uncertainty in the pixel depth of the image. The error of directly dividing the potential driving danger area of ​​the locomotive in the image without the help of features is large, which is prone to false detection and missed detection problems. Therefore, based on the scale information of each horizontal segment and the size information of the locomotive, the potential danger area is divided in the image to be detected, including: the size information of the locomotive at least includes the height information and width information of the locomotive, because the locomotive may be attached with cargo that exceeds the size of the locomotive, so the distance of the potential danger area in the real world is obtained based on the height and width of the locomotive itself and the preset buffer distance, and then based on the scale information of each horizontal segment, the distance of the potential danger area in the real world is converted to each horizontal segment of the image to be detected.

[0044] See also Figure 3 , Figure 3 is a schematic diagram of a straight track shown in an exemplary embodiment of the present application, such as Figure 3 As shown, if the current track is a straight track, the image to be detected is divided into a drivable area and a potential danger area. The drivable area is the area between the first track and the second track, and the potential danger area is the area divided in the image to be detected based on the scale information of each horizontal segment and the size information of the locomotive.

[0045] In one embodiment of the present application, before performing obstacle detection on an image to be detected based on a pre-trained obstacle detection model, the process also includes: obtaining historical locomotive driving images; labeling obstacles in the historical locomotive driving images based on obstacle labels to obtain an obstacle training set, wherein the obstacle labels include at least pedestrian labels, large obstacle labels, and small obstacle labels; and inputting the obstacle training set into a preset target detection for training to obtain an obstacle detection model.

[0046] In one embodiment of the present application, images or videos taken during the driving of a locomotive (such as a train, tram, etc.) are collected from a historical database as historical locomotive driving images. These images should cover different weather conditions, time periods, and possible obstacle types to ensure that the selected images are of good quality and high definition and can accurately reflect the real scene when the locomotive is driving. Select a tool suitable for image annotation, such as LabelImg, VIA (VGG Image Annotator) or other professional image annotation software. In the historical locomotive driving images, pedestrians are marked with pedestrian labels, large vehicles, buildings and bridges are marked with large obstacle labels, and roadblocks, small animals and small vehicles are marked with small obstacle labels. Other types of labels, such as traffic signs and road types, can also be added according to actual needs. The marked images are used as obstacle training sets, and the obstacle training sets are input into the target detection model for training. During the training process, the target detection model will learn to identify obstacles in the image and classify them according to the marked labels. The trained target detection model is exported to obtain an obstacle detection model.

[0047] Step S230: perform obstacle detection on the image to be detected based on the pre-trained obstacle detection model, obtain the area where the obstacle is located in the image to be detected according to the obstacle detection result, and issue an obstacle warning according to the area.

[0048] In one embodiment of the present application, performing obstacle warning according to the area in which the locomotive is located includes: if the area in which the locomotive is located is a potential danger area or a rail travel area, the locomotive performs obstacle warning and executes an emergency avoidance strategy.

[0049] In one embodiment of the present application, obtaining the area where the obstacle is located in the image to be detected according to the obstacle detection result includes: obtaining the pixel position of the obstacle in the image to be detected and the horizontal segment corresponding to the pixel position according to the obstacle detection result; obtaining the actual position of the obstacle relative to the locomotive based on the scale information of the horizontal segment corresponding to the pixel position and the pixel position of the obstacle; and providing auxiliary driving parameters for the emergency avoidance strategy according to the actual position of the obstacle relative to the locomotive.

[0050] In one embodiment of the present application, risk assessment is performed based on the obstacle type and the actual position of the obstacle relative to the locomotive in combination with the locomotive's own driving status (such as vehicle speed, acceleration, and steering speed, etc.). Emergency avoidance strategies include emergency avoidance, deceleration and stopping, or adjustment of driving trajectory, etc. The auxiliary driving parameters include at least braking parameters, steering parameters, or acceleration parameters. The braking parameters include calculating the braking force or braking time based on the emergency avoidance strategy and the actual position of the obstacle relative to the locomotive, to ensure that the locomotive can decelerate or stop smoothly and safely.

[0051] In one embodiment of the present application, after obstacle detection is performed on the image to be detected based on a preset target detection model, it also includes: obtaining the obstacle type of the obstacle and the area where the obstacle is located in the image to be detected according to the obstacle detection result; if the area where the obstacle is located is a rail running area, a first-level severe alarm is issued; if the obstacle type of the obstacle is a pedestrian, and the area where the obstacle is located is a potential danger area, a first-level severe alarm is issued; if the obstacle type of the obstacle is not a pedestrian, and the area where the obstacle is located is the potential danger area, a second-level moderate alarm is issued.

[0052] In one embodiment of the present application, the area where the obstacle is located is first determined. If the obstacle is in the rail travel area, no matter what the obstacle type is, as long as it is in the rail travel area, a level one serious alarm is immediately triggered, because the rail travel area is a key area for locomotive operation, and any obstacle may pose a danger to the locomotive operation. If the area where the obstacle is located is not in the rail travel area, it is further determined whether it is located in a potential danger area. If the obstacle is in a potential danger area and the obstacle type is a pedestrian, a level one serious alarm is also triggered. If the obstacle is in a potential danger area and the obstacle type is not a pedestrian, a level two moderate alarm is triggered. According to the triggered alarm level, take corresponding emergency measures. The level one serious alarm needs to be notified to the technician immediately, and it is determined whether it is necessary to perform operations such as immediate parking according to the on-site situation. The level two poisoning alarm can notify the technician, and emergency measures such as enhanced monitoring can be adopted.

[0053] See also Figure 4 , Figure 4The flowchart of the pedestrian detection method in the locomotive danger zone is shown in an exemplary embodiment of the present application. In one embodiment of the present application, before the pedestrian detection method is performed, the rail data set and the pedestrian data set are first collected to respectively train the semantic segmentation model and the target detection model for the semantic segmentation of the rail and the target detection of the pedestrian, and the rail data set is input, the rail data set includes historical locomotive driving images, the semantic segmentation model is trained based on the rail data set to obtain the rail semantic segmentation model, and the pedestrian data set is input to train the target detection model to obtain the pedestrian target detection model, and the pedestrian data set includes historical locomotive driving images and labels with pedestrian annotations. The pedestrian detection in the locomotive danger zone includes: inputting the image to be detected, performing real-time segmentation on the image to be detected based on the rail semantic segmentation model, extracting the rail driving area of ​​the current image according to the segmentation result and the locomotive driving path issued by the dispatching center, if the rail is in the driving area, obtaining the rail coordinates (x, y) of the rail driving area, segmenting the image to be detected horizontally, and fitting the track straight line y=ax+b of each segment in sequence, and obtaining the slope a of each horizontal segment. Then, the parallel tracks are obtained according to the a value of each horizontal segment, and the image pixel interval d between the parallel tracks is obtained. i , and assign a scale s to each transverse segmented rail according to the fixed rail spacing D in the actual scenario i , satisfying d i ×s i =D, where i is the number of horizontal segments. According to the scale information of each horizontal division, the dangerous area of ​​each horizontal segment is divided as a potential dangerous area. The target detection model detects pedestrians on the real-time data of the drone vehicle's onboard camera. If the pedestrian is in the potential dangerous area or the rail driving area, it is determined whether the pedestrian interferes with the driving, and an early warning is provided for the drone vehicle's automatic driving system. The scale information represented by each pixel in each horizontal segment area is combined with the number of pixels, and the pedestrian's actual relative camera position in the three-dimensional space is estimated according to the pixel position of the image. Coordinate conversion is performed to convert the relative camera position into the relative locomotive position, and the position parameter is provided to assist the automatic driving system.

[0054] See also Figure 5 , Figure 5 is a block diagram of a locomotive dangerous area obstacle detection device shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown is as follows. The device may also be applicable to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0055] like Figure 5 As shown, the exemplary locomotive dangerous area obstacle detection device includes:

[0056] The information input module 501 is used to obtain the image of the locomotive to be detected and the locomotive path information;

[0057] The area division module 502 is used to divide the rail travel area from the image to be detected based on the locomotive path information, and divide the potential danger area from the image to be detected according to the preset rail spacing;

[0058] The detection and warning module 503 is used to perform obstacle detection on the image to be detected based on the pre-trained obstacle detection model, obtain the area where the obstacle is located in the image to be detected according to the obstacle detection result, and issue an obstacle warning according to the area.

[0059] Figure 6 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0060] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603, such as executing the method described in the above embodiment. In the RAM 603, various programs and data required for system operation are also stored. The CPU 601, ROM 602 and RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0061] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0062] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, various functions defined in the system of the present application are executed.

[0063] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0064] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0065] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0066] Another aspect of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the above-mentioned locomotive dangerous area obstacle detection method. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0067] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the locomotive dangerous area obstacle detection method provided in each of the above embodiments.

[0068] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for detecting obstacles in a dangerous area of ​​a locomotive, characterized in that: The locomotive dangerous area obstacle detection method comprises: Obtaining the image of the locomotive to be detected and the locomotive path information; Based on the locomotive path information, a rail travel area is divided from the image to be detected, and a potential danger area is divided from the image to be detected according to a preset rail spacing; Obstacle detection is performed on the image to be detected based on a pre-trained obstacle detection model, the area where the obstacle is located in the image to be detected is obtained according to the obstacle detection result, and an obstacle warning is issued according to the area.

2. The method for detecting obstacles in a dangerous area of ​​a locomotive according to claim 1, characterized in that: Dividing the potential danger area from the image to be detected according to the preset rail scale includes: Establishing a coordinate system in the image to be detected, and extracting a plurality of track coordinate points in the rail travel area; The image to be detected is segmented horizontally, and track slopes of the first track and the second track in each horizontal segment are fitted based on the track coordinate points, wherein the image to be detected includes the first track and the second track; If the track slope difference between the first track and the second track in a horizontal segment is less than a preset difference threshold, obtaining an image pixel interval between the first track and the second track in the horizontal segment; Obtaining scale information of the one horizontal segment according to the preset rail spacing and the image pixel interval between the first rail and the second rail; A potential danger area is divided in the image to be detected based on the scale information of each lateral segment and the size information of the locomotive.

3. The method for detecting obstacles in a dangerous area of ​​a locomotive according to claim 1, characterized in that: Obstacle warning according to the area includes: If the area is the potential danger area or the rail travel area, the locomotive issues an obstacle warning and executes an emergency avoidance strategy.

4. The method for detecting obstacles in a dangerous area of ​​a locomotive according to claim 3, characterized in that: The area where the obstacle is located in the image to be detected obtained according to the obstacle detection result includes: Obtaining, according to the obstacle detection result, a pixel position of the obstacle in the image to be detected and a horizontal segment corresponding to the pixel position; Obtaining a real position of the obstacle relative to the locomotive based on the scale information of the lateral segment domain corresponding to the pixel position and the pixel position of the obstacle; Auxiliary driving parameters are provided for the emergency avoidance strategy according to the actual position of the obstacle relative to the locomotive.

5. The method for detecting obstacles in a dangerous area of ​​a locomotive according to claim 1, characterized in that: Dividing the rail travel area from the image to be detected based on the locomotive path information includes: Inputting the image to be detected into a pre-trained semantic segmentation model to perform semantic segmentation to extract rail features in the image to be detected; The rail features are matched with the locomotive path information, and if the match is successful, a rail travel area is divided from the image to be detected based on the rail features.

6. The method for detecting obstacles in a dangerous area of ​​a locomotive according to claim 1, characterized in that: Before performing obstacle detection on the image to be detected based on the pre-trained obstacle detection model, the method further includes: Obtain historical locomotive driving images; Annotating obstacles in the historical locomotive driving image based on obstacle labels to obtain an obstacle training set, wherein the obstacle labels at least include pedestrian labels, large obstacle labels, and small obstacle labels; The obstacle training set is input into a preset target detection for training to obtain an obstacle detection model.

7. The method for detecting obstacles in a dangerous area of ​​a locomotive according to claim 1, characterized in that: After performing obstacle detection on the image to be detected based on the preset target detection model, the method further includes: Obtaining the obstacle type of the obstacle and the area where the obstacle is located in the image to be detected according to the obstacle detection result; If the area where the obstacle is located is the rail travel area, a level one severe alarm is issued; If the obstacle type of the obstacle is a pedestrian, and the area where the obstacle is located is the potential danger area, a level one severe alarm is issued; If the obstacle type of the obstacle is not a pedestrian, and the area where the obstacle is located is the potential danger area, a second-level moderate alarm is issued.

8. A locomotive dangerous area obstacle detection device, characterized in that: The locomotive dangerous area obstacle detection device comprises: An information input module, used to obtain the image of the locomotive to be detected and the locomotive path information; A region division module, used to divide the rail travel region from the image to be detected based on the locomotive path information, and divide the potential danger region from the image to be detected according to a preset rail spacing; The detection and warning module is used to perform obstacle detection on the image to be detected based on a pre-trained obstacle detection model, obtain the area where the obstacle is located in the image to be detected according to the obstacle detection result, and issue an obstacle warning based on the area.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the locomotive dangerous area obstacle detection method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the method for detecting obstacles in a dangerous area of ​​a locomotive as claimed in any one of claims 1 to 7.

Images