[0043] In order to make the objects, technical solutions, and advantages of the present invention, the technical solutions in the present invention will be apparent from the drawings of the present invention, and it will be described in connection with the drawings of the present invention. , Not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained without creative labor are not made in the premise of creative labor.
[0044] In a traditional driving line, the train operation is completely dependent on the line closure and the driver's visual environment, or the driver's real-time observation and judgment of the driver's line is used, and the obstacle is discovered in advance, and the dispatch center is cleared in advance. Treatment, while the train can also take deceleration or braking.
[0045] Although the method of assisting the driver's obstacle detection by an optical image device can respond before hitting the obstacle, since the speed during the operation is very fast, the obstacle is detected to the driver The speed is high; and the above method can have a better detection effect for some color distinctive objects; however, for some of the obstacles similar to the surrounding environment, or the shape is easily confusing obstacles, for example, shapes and The stone, color and slate, the color and slate, etc. .
[0046]In addition, during the operation of the train automatic control system (CBTC) in the rail traffic, there may be an obstacle that affects the normal operation of the train on the train running line during the operation of natural, artificial or accidents. In this case, this is generally safe to ensure the driving of the train, but in the FAO (Fully Automatic Operation, all-automatic driving) driverless system, there is no driver's participation during the train operation, so it cannot be found in time to discover the obstacles on the line. . At the same time, the driver is manually judged that the obstacle may have a judgment error or not in time to discover obstacles, which affect the train efficiency and driving safety.
[0047] In response to the above, embodiments of the present invention provide a train operation method, figure 1 It is a flow chart of the train operation method provided by the embodiment of the present invention, such as figure 1 As shown, the method includes:
[0048] Step 110, acquire a plurality of multi-spectral cameras, a multi-spectral image collected separately, and a plurality of multi-spectral cameras are disposed at a trail at different perspectives;
[0049] Here, the multi-spectral camera is a sensor that collects multi-spectral images, can simultaneously acquire image data of N (n> 3), multi-spectral camera having multiple lenses, each lens for collecting a wavelength Spectral image. The multi-spectral image is a set of images composed of a grayscale image of the N (n> 3) spectral band.
[0050] It should be noted that the multi-spectral camera can not only collect the spectral image of the visible light band, but also collect the spectral image of the invisible light band, by analyzing the spectral characteristics of the object in different wavelengths in the multi-spectral image, and can determine the material of the object. .
[0051] Considering the perspective of the multi-spectral image acquired by a single multi-spectral camera is fixed, the obstacle is a stereo object, and the obstacle cannot be determined by a single angle multi-spectral image, and therefore, a plurality of multi-spectrum cameras are employed in the embodiment of the present invention. And the plurality of multi-spectral cameras are disposed at different viewing angles at different viewing angles, and multiple sets of multi-spectral images of different perspectives are collected by multiple multi-spectral cameras, and the obstacles can be determined by collecting individual multi-spectral images.
[0052] Step 120, perform image obstacle detection for each multi-spectral image, determine the obstacle position and candidate type of the target obstacle;
[0053] Here, the obstacle exists in the multi-spectral image; the image obstacle detection detects whether there is an obstacle in each multi-spectral image; the position of the obstacle position is the position of the target obstacle; the candidate type is an image barrier The type of the obtained target obstacle is detected.
[0054] Specifically, after the multi-spectral image acquired by the plurality of multi-spectral cameras, each multi-spectral image can be detected, and the target obstacle is present in each multi-spectral image. It should be noted that since the accuracy of the detection result of a single multi-spectral image is not high, the image detection result of the plurality of multi-spectral images in the present invention may determine whether or not there is a target obstacle.
[0055] When it is determined that there is a target obstacle, the obstacle position of the target obstacle is detected, and on this, the type of target obstacle belongs is determined according to the shape, size of the target obstacle in the multi-spectral image, and considers The image obstacle detected in step 120 detects the type of the target obstacle, which is determined in the case where the material is not considered, and therefore, the reliability and accuracy of the type of target obstacle obtained at this time. Lower, it is not suitable for the type of obstacle to the final target obstacle, so the type of target obstacle to which the image obstacle detection is detected is defined as the candidate type of the target obstacle.
[0056] Step 130 determines the material of the target obstacle based on the spectral data of the obstacle position based on each multi-spectral image;
[0057] Specifically, after the multi-spectral image collected by step 110, the material of the target obstacle can be determined according to the spectral data of the obstacle position of the target obstacle according to the multi-spectral image of each different band. The spectral data herein is mainly the spectral reflection characteristics of the object.
[0058] Since the spectral characteristics of the objects of different materials are different, the objects of different materials are different for radiation, absorption, transmission, and reflection characteristics of different wavelength electromagnetic waves (visible light, ultraviolet, infrared, microwave, etc.).
[0059] Taking the spectral reflection characteristics of the object as an example, for the same object, the reflectance varies with the wavelength of the wavelength; for the same wavelength, the reflection of the object of different materials is different.
[0060] Since the object of the different material is different from the spectrum characteristics of the light of different wavelengths, that is, the object is a unique spectral characteristic for the optical properties of the different wavelengths, and therefore, according to the spectral characteristics of the object, the material of the object can be determined, thereby achieving the classification of objects.
[0061] Step 140, based on the material to adjust the candidate type of the target obstacle to obtain an obstacle type of the target obstacle;
[0062] Specifically, after step 120 obtains the obstacle position and candidate type of the target obstacle, since the candidate type of the target obstacle cannot accurately characterize the real type of the target obstacle, further processing is required to determine the obstacle of the target obstacle. Type.
[0063] After step 130, the material of the target obstacle is determined, on the basis, according to the material of the target obstacle, the candidate type of the target obstacle obtained in step 120 is adjusted to determine the target obstacle. The type of obstacle, resulting in the obstacle type of the target obstacle can be more accurate to characterize the real type of target obstacle.
[0064] Step 150, based on the obstacle position of the target obstacle and the type of obstacle, the trachery is operated.
[0065] Specifically, after the above step is obtained by the obstacle position and the type of obstacle of the target obstacle, the control train can be performed according to the obstacle position and the type of obstacle of the target obstacle. For example, when the target obstacle is located outside the travel track of the train, it indicates that the target obstacle does not affect the safe operation of the train. At this time, the train can be operated normally; when the target obstacle is located on the trail, the target obstacle When the obstacle type is a slate, it indicates that the target obstacle has a greater impact on the safety operation of the train, and the train brake is required to be controlled or the notification dispatch center for cleavage processing.
[0066] The train operation method provided by the embodiment of the present invention, collects multi-spectral images by multiple spectrum images of a plurality of different perspectives, collecting a multi-spectral image, not only for image detection to locate the position of the target obstacle and determines its candidate type, It can be used to detect the material of the target obstacle, and detect by multi-spectral images of different perspectives, further improving the accuracy and reliability of the detection result; by analyzing the spectrum of the target obstacle in each multi-spectral image at different wavelengths Features, can determine the material of the target obstacle; combined with the material to adjust the candidate type, it can accurately determine the type of obstacle of the target obstacle, according to the obstacle position and obstacle type, control train, solve the traditional program need driver Artificial detection obstacles may have problems detected inaccurate, and improve the accuracy of detection.
[0067] Based on the above embodiment, step 120 includes:
[0068] The detection of various multi-spectral images is detected to obtain an obstacle area of each multi-spectral image;
[0069] The obstacle position and candidate type of the target obstacle is determined based on the impairment position of the multi-spectral image, and the obstacle area of the various multi-spectral images, and the obstacle position and candidate type of the target obstacle, and the type of candidate. The shape and / or volume of the target obstacle.
[0070] Specifically, after the multi-spectral image acquired by the plurality of multi-spectral cameras, each multi-spectral image can be detected to detect whether there is a target obstacle in each multi-spectral image, and there is a target barrier. In the case of the object, the region of the target obstacle in each multi-spectral image is determined, that is, the obstacle area of the target obstacle.
[0071] Subsequently, the obstacle position and candidate type of the target obstacle can be determined according to the various multi-spectral images and the impairment position of the multi-spectral camera of each multi-spectral image, and the obstacle area of the target obstacle in each multi-spectral image; The obstacle position, obstacle type, and target disorder of the target obstacle can be determined according to the various multi-spectral images and the impairment position of the multi-spectral camera of each multi-spectral image, and the obstacle area of the target obstacle in each multi-spectral image. The shape of the object; or determines the obstacle position, obstacle of the target obstacle, the obstacle area of the target obstacle in each multi-spectral image and the captain of the multi-spectral image of each multi-spectral image, and the obstacle area of the target obstacle in each multi-spectral image. The volume of the target obstacle.
[0072] It is also possible to determine the obstacle position, obstacle type, and target of the target obstacle, the target obstacle area, the obstacle area, the target of the target obstacle, and the obstacle area of the target obstacle in each multi-spectral image, and the obstacle area of the target obstacle in each multi-spectral image. The shape of the obstacle and the volume of the target obstacle.
[0073] The shooting position of the multi-spectral camera includes a photographing position and a shooting angle of the multi-spectral camera. Take three multi-spectral cameras as an example, figure 2 It is a schematic diagram of a photographing position of the multi-spectral camera provided by the embodiment of the present invention, such as figure 2 As shown, three multi-spectral cameras collect multi-spectral images of target obstacles in different locations and perspectives.
[0074] Based on the above embodiment, step 140 includes:
[0075] Based on the material of the target obstacle, the shape of the target obstacle is adjusted, and the type of obstacle is adjusted to obtain an obstacle type of the target obstacle.
[0076] Specifically, based on the candidate type, material, and shape and / or volume of the target obstacle, the present invention can be selected according to the material of the target obstacle, the shape of the target obstacle, according to the shape of the target obstacle. Types are adjusted to obtain an obstacle type of target obstacles.
[0077] It is also possible to adjust the candidate type of the target obstacle according to the material of the target obstacle, and the type of obstacle to the target obstacle can also be adjusted to obtain the type of obstacle of the target obstacle; the shape of the target obstacle, the shape of the target obstacle and the target The volume of the obstacle, adjusts the candidate type of the target obstacle to determine the type of obstacle of the target obstacle.
[0078] The method provided in the embodiment of the present invention, combines the material, shape, and / or volume of the target obstacle, and adjusts the candidate type of the target obstacle to determine the type of obstacle of the target obstacle.
[0079] Based on the above embodiment, in step 120, the obstacle position and candidate type of the target obstacle is determined based on the imaging position postage of the multi-spectral image, and the obstacle area of the respective multi-spectral images, or the target obstacle. The obstacle position and candidate type, as well as the shape and / or volume of the target obstacle, including:
[0080]If the target obstacle is on the track, the obstacle position and candidate type of the target obstacle is determined based on the imaging position postage of the multi-spectral image, and the obstacle area of the respective multi-spectral images, or the target obstacle is determined. The obstacle position and candidate type, as well as the shape and / or volume of the target obstacle;
[0081] Among them, whether the target obstacle is in the track is determined by the positional relationship between the obstacle area of each multi-spectral image and the track region.
[0082] Specifically, the image barrier region is detected on the respective multi-spectral images to obtain an obstacle area of the target obstacle, the trailer region of the target obstacle in each multi-spectral image and the track area of the train travel track. The positional relationship is determined whether the target obstacle is in the traveling track of the train. If the target obstacle is on the travel rail of the train, the shooting position of the multi-spectral image of each multi-spectral image can be taken according to the various multi-spectral images and the multi-spectral camera of each multi-spectral image. As well as the obstacle area of the target obstacle in each multi-spectral image, the obstacle position and candidate type of the target obstacle are determined.
[0083] It is also possible to determine the obstacle position, obstacle type, and target of the target obstacle, the obstacle area of the target obstacle, and the obstacle area of the target obstacle in each multi-spectral image, and the obstacle area of the target obstacle in each multi-spectral image. The shape of the obstacle; or the obstacle place, obstacle to the target obstacle is determined according to the various multi-spectral images and the impairment of the multi-spectral camera of each multi-spectral image, and the obstacle area of the target obstacle in each multi-spectral image. The volume of the object and the volume of target obstacles.
[0084] It is also possible to determine the obstacle position, obstacle type, and target of the target obstacle, the target obstacle area, the obstacle area, the target of the target obstacle, and the obstacle area of the target obstacle in each multi-spectral image, and the obstacle area of the target obstacle in each multi-spectral image. The shape of the obstacle and the volume of the target obstacle.
[0085] Based on the above embodiment, step 150 includes:
[0086] Determining the threat level of target obstacles based on obstacle position and obstacle type of target obstacle;
[0087] Control the train operation based on the threat level of the target obstacle.
[0088] The threat level here indicates that the influence of target obstacles for train safety operation, the higher the threat level, the greater the impact on train safety operation; in turn, the lower the threat level, the smaller the impact on the safe operation of the train.
[0089] Considering that the extent of the target obstacle is determined whether the train can operate normally, after obtaining the obstacle position of the target obstacle and the type of obstacle, the target barrier can be determined according to the type of obstacle and obstacle. For the threat level of the train.
[0090] Thereafter, the corresponding operation can be performed according to the threat level control train. For example, if the target obstacle is a slate on the traveling track of the train, it is determined that the target obstacle has a greater impact on the safety operation of the train, can control the train brake or the notification dispatch center for cleavage processing; if the target obstacle is The plastic bag on the travel track of the train is determined that the target obstacle has a small impact on the safety operation of the train, and the train can be controlled normally.
[0091] Based on the above embodiment, in step 150, the threat level of the target obstacle is determined based on the obstacle position and the type of obstacle of the target obstacle, including:
[0092] Based on the obstacle position and obstacle type of the target obstacle, the speed and / or braking performance of the train, determines the threat level of the target obstacle.
[0093] Since the same target obstacle is different from the threat level of the train traveling at different speeds, the higher the train speed, the higher the threat level of the train; it is, the slower the trains, the target obstacle is for the train. The lower the threat level. Similarly, the braking performance of the train will also affect the target obstacle to the threat level of the train. The better the train, the more the train is likely to park before the obstacle of the target obstacle, the target obstacle for the train The lower the threat level; in turn, the worse the brake performance of the train, the lower the trachee, the lower the probability of parking before the obstacle position of the target obstacle, the higher the target obstacle to the threat level of the train.
[0094] Determine the target obstacle on the clutter of the train according to the target obstacle, the target obstacle can also be combined with the speed of the train to determine the threat of the train for the train; or obstacles to the target obstacle Location, obstacle type, and braking performance of the train, determine the threat of the trachee for the train.
[0095] It is also possible to determine the threat of the train to determine the threat of the train according to the obstacle position, obstacle type, and the speed of the trachee.
[0096] image 3 It is an overall flow chart of the train operation method provided by the embodiment of the present invention, such as image 3 As shown, the method includes:
[0097] Step 310, set a plurality of multi-spectral cameras at different viewing angles at a head of the train, acquire multiple multi-spectral images collected by multiple multi-spectral cameras;
[0098] Step 320, perform image obstacles for each multi-spectral image, and detects whether there is a target obstacle;
[0099] Step 330, if there is a target obstacle, and the target obstacle is on the train track, step 340 and step 350 are performed; otherwise, step 390 is performed;
[0100] Step 340: Determine the obstacle area of the target obstacle in each multi-spectral image;
[0101] Step 341: Determining the obstacle position and candidate type of the target obstacle based on the shooting position postage of the multi-spectral image, and the obstacle area of the respective multi-spectral images, or the obstacle position and candidate of the target obstacle Type, and shape and / or volume of target obstacles;
[0102] Step 350 determines the material of the target obstacle based on the spectral data of each multi-spectral image in the obstacle position;
[0103] Step 360, based on the material of the target obstacle, and the shape and / or volume of the target obstacle, the type of obstacle is adjusted to obtain an obstacle type of the target obstacle;
[0104] Step 370, based on the obstacle position and the type of obstacle, and the speed and / or braking performance of the train, the threat level of the target obstacle is determined;
[0105] Step 380 controls the train operation based on the threat level of the target obstacle;
[0106] Step 390, the control train is operating normally.
[0107] Next, the train operating device provided by the present invention will be described, and the train operation method described below can correspond to each other.
[0108] Figure 4 It is a schematic structural diagram of the train operating device provided by the present invention, such as Figure 4 As shown, the apparatus includes:
[0109] The acquisition unit 410 is used to acquire a plurality of multi-spectral cameras to be collected, respectively, and the plurality of multi-spectral cameras are disposed at different perspectives;
[0110] The image detecting unit 420 is configured to detect the image barrier detection of each multi-spectral image, determine the obstacle position and candidate type of the target obstacle;
[0111] The material determination unit 430 is configured to determine the material of the target obstacle based on the spectral data of each multi-spectral image in the obstacle position;
[0112] Type determining unit 440, for adjusting the candidate type of the target obstacle based on the material to obtain an obstacle type of the target obstacle;
[0113] The control unit 450 controls the train operation based on the obstacle position and the type of obstacle of the target obstacle.
[0114] The train operating device provided by the present invention collects a multi-spectral image by a plurality of different viewing angles, and the acquired multi-spectral image can be used not only for image detection to locate the position of the target obstacle and determine its candidate type, can also be used The material of the target obstacle is detected, and the multi-spectral image of multiple different perspectives is detected, and the accuracy and reliability of the detection result is further improved; the spectral characteristics of the target obstacle in each multi-spectral image are analyzed. It is possible to determine the material of the target obstacle; combined with the material to adjust the candidate type, it can accurately determine the type of obstacle of the target obstacle, according to the obstacle position and the type of obstacle type, the tradition, the traditional program can be used to manually The detection obstacle has an inaccurate problem, and improves the accuracy of the test.
[0115] Based on the above embodiment, the image detecting unit 420 is for:
[0116] The detection of various multi-spectral images is detected to obtain an obstacle area of each multi-spectral image;
[0117] The obstacle position and candidate type of the target obstacle is determined based on the imaging position position of the multi-spectral image corresponding to the photographing position of the multi-spectral camera, and the obstacle area of the respective multi-spectral images, or the position of the target obstacle, or the obstacle position of the target obstacle. Candidate types, and shape and / or volume of the target obstacle.
[0118] Based on the above embodiment, the type determination unit 440 is used in:
[0119] Based on the material of the target obstacle, the type of obstacle is adjusted to obtain an obstacle type of the target obstacle.
[0120] Based on the above embodiment, the apparatus further includes a position determining unit for:
[0121] If the target obstacle is on the track, the obstacle position and candidate type of the target obstacle are determined based on the imaging position postage of the multi-spectral image, and the obstacle area of the respective multi-spectral images, or Determine the obstacle position and candidate type of the target obstacle, and the shape and / or volume of the target obstacle;
[0122] Among them, whether the target obstacle is in the track is determined based on the positional relationship between the obstacle area of each multi-spectral image and the track region.
[0123] Based on the above embodiment, the control unit 450 is for:
[0124] Based on the obstacle position and obstacle type of the target obstacle, the threat level of the target obstacle is determined;
[0125] Controls the train operation based on the threat level of the target obstacle.
[0126] Based on the above embodiment, the apparatus also includes a threat level determining unit for:
[0127] The threat level of the target obstacle is determined based on the obstacle position and the type of obstacle of the target obstacle, and the speed and / or braking performance of the train.
[0128] Figure 5 Example of an entity structure schematic of an electronic device, such asFigure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 pass the communication bus 540. Complete communication between each other. Processor 510 can call logic instructions in memory 530 to perform train operation methods, including: acquiring multi-spectral images collected by multiple multi-spectral cameras, the plurality of multi-spectral cameras are disposed at different perspectives at different perspectives At; image obstacle detection of each multi-spectral image, determining the obstacle position and candidate type of the target obstacle; the material of the target obstacle is determined based on the spectral data of each multi-spectral image in the obstacle position; The material adjusts the type of candidate of the target obstacle to obtain an obstacle type of the target obstacle; controlling the train operation based on an obstacle position and an obstacle type of the target obstacle.
[0129] Further, the logic instructions in the above-described memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when the software functional unit is sold or used. Based on this understanding, the technical solution of the present invention essentially ors a portion of the prior art or a portion of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or some steps of the method of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, RAD-ONLY MEMORY), RAM, RAM, RANDOM Access Memory, disk, or disc or optical disk, etc. can store program code .
[0130] On the other hand, the present invention also provides a computer program product that includes a computer program stored on a non-transitory computer readable storage medium, the computer program including program instructions, when the program instruction is computer When executed, the computer can perform the train operation method provided by each method described above, including: acquiring a plurality of multi-spectral cameras, a multi-spectral image collected, respectively, the plurality of multi-spectral cameras being disposed at different perspectives; Image obstacle detection of each multi-spectral image is determined to determine the obstacle position and candidate type of the target obstacle; the material of the target obstacle is determined based on the spectral data of each multi-spectral image in the obstacle position; based on the Material Adjusts the candidate type of the target obstacle to obtain an obstacle type of the target obstacle; control the train operation based on the obstacle position and the type of obstacle of the target obstacle.
[0131] Yet another aspect, the present invention also provides a non-transitory computer readable storage medium, which stores a computer program that is implemented when executed by the processor to perform the above-described train operation, including: acquisition Multi-spectral images collected by multiple multi-spectral cameras, the plurality of multi-spectral cameras are disposed at different viewing angles; detecting individual multi-spectral images, determining obstacles and candidates for target obstacles Type; the material of the target obstacle is determined based on the spectral data of each multi-spectral image in the obstacle position; the type of candidate type of the target obstacle is adjusted based on the material to obtain the type of obstacle of the target obstacle. The train operation is controlled based on an obstacle position and an obstacle type of the target obstacle.
[0132] The device embodiment described above is merely schematic, wherein the unit as the separation component may be or may not be physically separated, and the components displayed as the unit may or may also be not a physical unit, that is, it can be located One place or can also be distributed to multiple network elements. The object of the present embodiment can be implemented in accordance with the actual needs of the actual needs. One of ordinary skill in the art will understand and implement in the case where there is no creative labor.
[0133] Through the description of the above embodiments, those skilled in the art will clearly understand that the embodiments can be implemented by means of software plus necessary, of course, can, of course, can be passed through hardware. Based on this, the above technical solution essentially or contributes to the prior art, the computer software product can be stored in a computer readable storage medium, such as ROM / RAM, magnetic Disc, optical disc, etc., including several instructions to enable a computer device (can be a personal computer, server, or network device, etc.) to perform various embodiments of certain portions of various embodiments or embodiments.
[0134] It will be noted in that the above embodiment is intended to illustrate the technical solutions of the present invention, not to limit the invention; The technical solutions described in the foregoing examples are modified, or part of the technical features in which these modifications or replacements do not allow the nature of the corresponding technical solutions from the spirit and scope of the technical solutions of the present invention.
