Control method and device of highway-railway combined transport vehicle

Abstract

The invention provides a control method and device for a highway-railway combined transport vehicle, and relates to the technical field of highway-railway combined transport. The method comprises the steps that after the highway-railway combined transport vehicle travels to a preset area of a railway station, a depth image shot by the 3D camera is acquired; determining a first linear equation of a first straight line corresponding to the railway steel rail in the image coordinate system of the depth image; determining a second linear equation of a second straight line corresponding to the railway steel wheel in the image coordinate system; and on the basis of the first linear equation and the second linear equation, the highway-railway combined transport vehicle is controlled to carry out revolution railway operation. According to the control method and device of the highway-railway combined transportation vehicle, the labor cost of revolution and railway transportation operation can be reduced, the conversion efficiency is improved, and the advantages of highway-railway combined transportation are fully played.

Description

Technical Field

[0001] The present application relates to the technical field of road-rail combined transport, and in particular to a control method and device for a road-rail combined transport vehicle. Background Art

[0002] The current logistics industry is in a state of rapid development, but the proportion of logistics costs is obviously high, and logistics costs remain high. Among them, the high cost of road-to-rail operations is an important factor. Road-to-rail operations are a form of road-rail combined transport. As an ideal and efficient "door-to-door" multimodal transport mode, road-rail combined transport can organically integrate traditional, single road and rail transport through resource integration, giving full play to the advantages of long-distance, large-volume, low-cost railway transport and flexible, "door-to-door" transportation of road transport. In order to adapt to the development of road-rail combined transport, road-rail combined transport vehicles came into being, but there is currently no intelligent control method based on road-rail combined transport vehicles, resulting in low efficiency and high cost of road-rail operations, or road-rail conversion. Summary of the invention

[0003] The present application provides a control method and device for a road-rail combined transport vehicle, which is used to solve the defects of low road-rail conversion efficiency and high cost in the prior art, thereby improving the road-rail conversion efficiency and reducing the road-rail conversion cost.

[0004] In a first aspect, the present application provides a control method for a road-rail transport vehicle, the road-rail transport vehicle comprising a 3D camera and a plurality of railway steel wheels, the 3D camera being disposed at the bottom of the road-rail transport vehicle, the field of view of the 3D camera facing the ground and covering a portion of one of the railway steel wheels; the method comprising:

[0005] After the road-rail transport vehicle travels to a preset area of ​​a railway station, acquiring a depth image captured by the 3D camera;

[0006] Determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system;

[0007] Based on the first straight line equation and the second straight line equation, the road-rail combined transport vehicle is controlled to perform road-to-rail operations.

[0008] Optionally, controlling the road-rail combined transport vehicle to perform a road-to-rail operation based on the first straight line equation and the second straight line equation includes:

[0009] Determine an angle between the first straight line and the second straight line based on the first straight line equation and the second straight line equation;

[0010] Based on the size of the angle, the road-rail combined transport vehicle is controlled to perform road-to-rail operations.

[0011] Optionally, controlling the road-rail combined transport vehicle to perform a road-to-rail operation based on the size of the included angle includes:

[0012] When the size of the angle is within a preset angle range, determining a first distance between the first straight line and the second straight line in the image coordinate system based on the first straight line equation and the second straight line equation;

[0013] Based on the first spacing, controlling the road-rail combined transport vehicle to complete the road-to-rail operation;

[0014] or:

[0015] When the size of the angle is not within the preset angle range, outputting a prompt message to prompt the staff to adjust the position of the road-rail combined transport vehicle so that the size of the angle is within the preset angle range;

[0016] After the size of the angle is within the preset angle range, re-determining the first straight line equation and the second straight line equation;

[0017] Determine the first distance between the first straight line and the second straight line in the image coordinate system based on the re-determined first straight line equation and the second straight line equation;

[0018] Based on the first spacing, the road-rail combined transport vehicle is controlled to complete the road-to-rail operation.

[0019] Optionally, controlling the road-rail combined transport vehicle to complete the road-to-rail operation based on the first spacing includes:

[0020] Based on the first spacing, determining a second spacing; the second spacing is the actual spacing between the railway rail and the railway steel wheel;

[0021] Based on the second spacing, the road-rail combined transport vehicle is controlled to complete the road-to-rail operation.

[0022] Optionally, the determining a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image includes:

[0023] Removing the pixel points corresponding to the railway steel wheel in the depth image to obtain a target image;

[0024] The first straight line equation is determined based on the value of each pixel in the target image.

[0025] Optionally, determining the first straight line equation based on the value of each pixel in the target image includes:

[0026] Calculate the difference between the values ​​of two adjacent pixels in each row of the target image to obtain a deviation value;

[0027] Determine the target pixel based on the size of the deviation value;

[0028] Perform linear fitting on the target pixel points to obtain the first straight line equation.

[0029] Optionally, the determining a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system includes:

[0030] Determine the coordinate value corresponding to the center line of the railway steel wheel covered by the field of view of the 3D camera in the image coordinate system;

[0031] Based on the coordinate values, the second straight line equation is determined.

[0032] In a second aspect, the present application further provides a control device for a road-rail transport vehicle, the road-rail transport vehicle comprising a 3D camera and a plurality of railway steel wheels, the 3D camera being arranged at the bottom of the road-rail transport vehicle, the field of view of the 3D camera being towards the ground and covering a portion of one of the railway steel wheels; the device comprising:

[0033] An acquisition module, used for acquiring a depth image taken by the 3D camera after the road-rail combined transport vehicle travels to a preset area of ​​a railway station;

[0034] A determination module, configured to determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system;

[0035] An operation module is used to control the road-rail combined transport vehicle to perform road-to-rail operations based on the first straight line equation and the second straight line equation.

[0036] In a third aspect, the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0037] In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0038] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0039] The control method and device of the road-rail combined transport vehicle provided in the present application obtains a depth image taken by a 3D camera and determines the straight line equations corresponding to the railway rails and the railway wheels in the depth image. Then, the road-to-rail operation process of the road-rail combined transport vehicle can be intelligently guided according to the two straight line equations, thereby realizing the road-to-rail operation of the road-rail combined transport vehicle in an unmanned command mode, reducing the work intensity of personnel, reducing the labor cost of the road-to-rail operation, and improving the conversion efficiency. It can give full play to the advantages of road-rail combined transport and promote the development of the logistics industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 It is a flowchart of a control method of a road-rail combined transport vehicle provided in an embodiment of the present application;

[0042] Figure 2 is a structural schematic diagram of a road-rail combined transport vehicle provided in an embodiment of the present application;

[0043] Figure 3 It is a schematic diagram of the operating principle of the road-rail combined transport vehicle provided in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a depth image provided by an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of the depth image processing process provided by an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of the structure of a control device for a road-rail combined transport vehicle provided in an embodiment of the present application;

[0047] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] Figure 1is a flow chart of a control method for a road-rail combined transport vehicle provided in an embodiment of the present application. Figure 1 An embodiment of the present application provides a control method for a road-rail transport vehicle, which includes a 3D camera and multiple railway steel wheels. The 3D camera is arranged at the bottom of the road-rail transport vehicle, and the field of view of the 3D camera is toward the ground and covers a portion of a railway steel wheel. Figure 2 is a schematic diagram of the structure of the road-rail transport vehicle provided in an embodiment of the present application. The execution subject of the control method of the road-rail transport vehicle may be an electronic device, for example, a controller. The following description is made by taking the execution subject of the method as a controller as an example. The method may include:

[0050] Step 110: After the road-rail transport vehicle travels to a preset area of ​​a railway station, a depth image captured by a 3D camera is acquired;

[0051] Step 120, determining a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determining a second straight line equation of a second straight line corresponding to the railway wheel in the image coordinate system;

[0052] Step 130: Based on the first straight line equation and the second straight line equation, control the road-rail combined transport vehicle to perform road-to-rail operation.

[0053] Figure 3 Schematic diagram of the operation principle of the road-rail transport vehicle provided in the embodiment of the present application. Figure 3 As shown, the road-rail transport vehicle is a dual-purpose vehicle that can travel on both roads and railways. When the road-rail transport vehicle is traveling on the road, it is towed by the truck head, and the lifting and suspension device of the road-rail transport vehicle lowers the vehicle wheels to the lowest level. At this time, the railway steel wheels of the road-rail transport vehicle are lifted, and the road-rail transport vehicle can travel on the road. When the road-rail transport vehicle needs to travel on the railway, the road-rail transport vehicle travels to the railway station, and the truck driver aligns the railway steel wheels of the road-rail transport vehicle with the railway rails on the ground. Then the lifting and suspension device of the road-rail transport vehicle raises the vehicle wheels to the highest level. At this time, the railway steel wheels of the road-rail transport vehicle can fall on the railway rails on the ground, and the truck head is uncoupled from the road-rail transport vehicle, and the road-rail transport vehicle can be marshaled and travel on the railway. The road-to-rail operation needs to be carried out in the preset area of ​​the railway station. The railway rails in this area are raised on a flat hard ground.

[0054] In step 110, after the road-rail transport vehicle travels to the preset area of ​​the railway station, the road-rail operation begins. At this time, the controller can obtain the depth image taken by the 3D camera. The depth image includes part of the railway steel wheel and part of the railway rail.

[0055] In step 120, the controller may determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway wheel in the image coordinate system. It can be understood that the positions of the first straight line and the second straight line are the positions of the railway rail and the railway wheel in the depth image.

[0056] In step 130, the controller can determine the relative positions of the railway rails and the railway wheels based on the first straight line equation and the second straight line equation, and guide the road-rail combined transport vehicle to perform road-to-rail operations based on the relative positions.

[0057] The control method of the road-rail combined transport vehicle provided in the embodiment of the present application obtains a depth image taken by a 3D camera and determines the straight line equations corresponding to the railway rails and the railway wheels in the depth image. Then, the road-to-rail operation process of the road-rail combined transport vehicle can be intelligently guided according to the two straight line equations, thereby realizing the road-to-rail operation of the road-rail combined transport vehicle in an unmanned command mode, reducing the work intensity of personnel, reducing the labor cost of the road-to-rail operation, and improving the conversion efficiency. It can give full play to the advantages of road-rail combined transport and promote the development of the logistics industry.

[0058] In one embodiment, based on the first straight line equation and the second straight line equation, the road-rail combined transport vehicle is controlled to perform the road-to-rail operation, including: based on the first straight line equation and the second straight line equation, determining the angle between the first straight line and the second straight line; based on the size of the angle, controlling the road-rail combined transport vehicle to perform the road-to-rail operation.

[0059] Further, in one embodiment, based on the size of the angle, the road-rail combined transport vehicle is controlled to perform the road-to-rail operation, including: when the size of the angle is within a preset angle range, based on the first straight line equation and the second straight line equation, determining a first spacing between the first straight line and the second straight line in the image coordinate system; based on the first spacing, controlling the road-rail combined transport vehicle to complete the road-to-rail operation; or: when the size of the angle is not within the preset angle range, outputting a prompt message to prompt the staff to adjust the position of the road-rail combined transport vehicle so that the size of the angle is within the preset angle range; after the size of the angle is within the preset angle range, redetermining the first straight line equation and the second straight line equation; based on the redetermined first straight line equation and second straight line equation, determining a first spacing between the first straight line and the second straight line in the image coordinate system; based on the first spacing, controlling the road-rail combined transport vehicle to complete the road-to-rail operation.

[0060] In this application, the 3D camera, controller and wireless communication device can be integrated into a design, and the whole is packaged into a system module box and installed at the bottom of the vehicle. The 3D camera is used to collect depth images. The controller is used to process the depth image and generate the relative position data of the railway steel wheels of the road-rail combined transport vehicle and the railway rails on the ground. The wireless communication device transmits the relative position data to the mobile phone APP application at the upper end, and displays it on the APP display system. When the staff opens the mobile phone APP application and connects to the wireless, the relative position data of the railway steel wheels of the road-rail combined transport vehicle and the railway rails on the ground can be displayed in real time, guiding the staff to drive the road-rail combined transport vehicle to the appropriate drop-off position, thereby improving the efficiency of the entire road-to-rail reloading operation.

[0061] Specifically, when the road-to-rail operation is in progress, the road-rail transport vehicle is connected to the truck head, and the system module box is powered by the truck battery. Since the 3D camera, system module box, and railway steel wheels are all fixedly installed on the road-rail transport vehicle, the positions of the three are relatively fixed. During the road-to-rail operation of the road-rail transport vehicle, the positions of the railway rails and railway steel wheels within the field of view of the 3D camera are relatively changed. Through algorithm processing and calculation by the controller, the relative position data of the two (such as relative offset and angle data) can be obtained, and sent to the staff's mobile phone APP application through a wireless communication device to guide the road-rail transport vehicle to perform the road-to-rail operation.

[0062] The controller can determine the angle between the first straight line and the second straight line through the first straight line equation and the second straight line equation. Specifically, the controller can obtain the slopes of the two straight lines (the first straight line and the second straight line) through the first straight line equation and the second straight line equation, and calculate the angle between the two straight lines based on the slopes.

[0063] When the angle between the first straight line and the second straight line is within a preset angle range (eg, -3° to 3°), the two straight lines can be considered parallel. At this time, the position of the road-rail transport vehicle can be adjusted according to the first spacing between the two straight lines to complete the road-rail operation.

[0064] If the angle between the first straight line and the second straight line is not within the preset angle range (for example: -3°-3°), it means that the two straight lines are not parallel. At this time, the controller can send a prompt message to the staff's mobile phone APP application through a wireless communication device, prompting the staff to adjust the position of the road-rail transport vehicle so that the angle is within the preset angle range. After adjustment, the two straight lines are parallel. The controller can guide the road-rail transport vehicle to perform road-to-rail operations by re-determining the first straight line equation and the second straight line equation to calculate the first spacing between the two straight lines.

[0065] The control method of the road-rail combined transport vehicle provided in the embodiment of the present application determines the angle between the first straight line and the second straight line based on the equation of the first straight line and the equation of the second straight line, and adjusts the position of the road-rail combined transport vehicle when the angle is not within the preset angle range, so that the first straight line is parallel to the second straight line, that is, the railway steel wheels of the road-rail combined transport vehicle are parallel to the railway rails on the ground, and then completes the road-to-rail operation through the first spacing guidance, thereby realizing the road-to-rail operation in the unmanned command mode and improving the conversion efficiency.

[0066] In one embodiment, based on the first spacing, controlling the road-rail combined transport vehicle to complete the road-to-rail operation includes: determining the second spacing based on the first spacing; the second spacing is the actual spacing between the railway rails and the railway steel wheels; based on the second spacing, controlling the road-rail combined transport vehicle to complete the road-to-rail operation.

[0067] Specifically, according to the camera imaging principle, the second spacing is calculated as follows:

[0068] D=L×H / d

[0069] Wherein, D represents the second distance, L represents the first distance, H represents the height of the 3D camera from the ground, and d represents the focal length of the 3D camera.

[0070] The first spacing is the distance between the railway rail and the railway steel wheel in the image coordinate system of the depth image, and the second spacing is the actual spacing between the railway rail and the railway steel wheel.

[0071] After the controller calculates the second distance, it can send the second distance to the staff's mobile phone APP application. The staff can adjust the position of the road-rail transport vehicle based on the received second distance so that the railway steel wheels of the road-rail transport vehicle are fully aligned with the railway rails on the ground, thereby completing the road-to-rail operation.

[0072] The control method of the road-rail combined transport vehicle provided in the embodiment of the present application calculates the second spacing through the first spacing, so as to determine the distance between the railway steel wheel and the railway rail, thereby adjusting the position of the road-rail combined transport vehicle and completing the road-to-rail operation, thereby improving the intelligence of the road-to-rail operation, reducing the labor cost of the road-to-rail operation, improving the conversion efficiency, and giving full play to the advantages of road-rail combined transport.

[0073] In one embodiment, determining a first straight line equation of a first straight line corresponding to a railway rail in an image coordinate system of a depth image includes: removing pixel points corresponding to railway wheels in the depth image to obtain a target image; and determining the first straight line equation based on the value of each pixel point in the target image.

[0074] Furthermore, in one embodiment, based on the value of each pixel in the target image, a first straight line equation is determined, including: performing difference calculation on the values ​​of two adjacent pixel points in each row of the target image to obtain a deviation value; determining the target pixel point based on the size of the deviation value; and performing linear fitting on the target pixel point to obtain the first straight line equation.

[0075] Figure 4 is a schematic diagram of a depth image provided by an embodiment of the present application. Figure 4 Since the field of view of the 3D camera covers part of the railway steel wheel, there are pixels corresponding to the railway steel wheel in the depth image. The controller can remove the pixels corresponding to the railway steel wheel in the depth image to obtain the target image. At this time, the target image includes the ground and the railway rails. The controller can determine the first straight line equation based on the value of each pixel in the target image.

[0076] The value of each pixel of the depth image represents the height from that point to the camera. The value of each pixel is within the range of 0 to 65535. The ground is flat, and the railroad rails protrude from the ground by about 30 mm. Therefore, when the deviation value of two adjacent pixels in each row of the target image is greater than the first preset value (such as 20) and less than the second preset value (such as 40), it can be determined that there is a pixel representing the railroad rail in these two pixels, that is, the target pixel. Specifically, the point with the smaller height value from the camera among the two pixels is the target pixel. The controller can obtain the first straight line equation by performing linear fitting on all target pixels.

[0077] In other words, the depth image can be first set with a region of interest (ROI) to obtain the target image. Specifically, the image after removing the railway steel wheel can be used as the target image. The target image is a two-dimensional array A(m,n), where m is the number of array columns and n is the number of rows. The target image is divided into n one-dimensional arrays according to the number of rows, which can be recorded as A1 = {d11, d12, ... d1m}, A2 = {d21, d22, ... d2m}, ... An = {dn1, dn2, ... dnm}. Taking array A1 as an example, the values ​​of two adjacent pixels are calculated by difference, for example, the value of the latter pixel is subtracted from the value of the previous pixel. If the obtained deviation value is greater than 20 and less than 40, it means that the latter pixel is a point of the railway rail in the image coordinate system of the target image. By comparing row by row in this way, a series of target pixel points of the railway rail can be obtained. Figure 5 : is a schematic diagram of the depth image processing process provided by the embodiment of the present application. The target pixel point is also the edge coordinate point in the figure. The auxiliary line is the center line of the railway steel wheel in the depth image. The first straight line equation of the railway rail can be obtained by linearly fitting the target pixel point.

[0078] The control method of the road-rail combined transport vehicle provided in the embodiment of the present application calculates the deviation value of two adjacent pixel points in each row of the target image, and determines the target pixel point based on the deviation value, thereby obtaining the first straight line equation, providing data preparation for guiding the road-to-rail operation, improving the intelligence of the road-to-rail operation, and improving the conversion efficiency.

[0079] In one embodiment, determining a second straight line equation of a second straight line corresponding to a railway steel wheel in an image coordinate system includes: determining a coordinate value corresponding to a center line of the railway steel wheel covered by a field of view of a 3D camera in the image coordinate system; and determining a second straight line equation based on the coordinate value.

[0080] Since the 3D camera and the railway steel wheel are both fixedly installed on the road-rail combined transport vehicle, the position of the steel railway wheel on the depth image is fixed, and its straight line equation, that is, the second straight line equation, is also unchanged in the image coordinate system.

[0081] like Figure 5 As shown, the second straight line corresponding to the center line of the railway steel wheel covered by the field of view of the 3D camera in the image coordinate system is determined, multiple points are randomly selected in the second straight line, and the coordinate values ​​corresponding to the multiple points are obtained, and then the equation of the second straight line can be determined based on these coordinate values.

[0082] The control method of the road-rail combined transport vehicle provided in the embodiment of the present application determines the coordinate value corresponding to the center line of the railway steel wheel covered by the field of view of the 3D camera in the image coordinate system, determines the second straight line equation, provides data preparation for guiding the road-to-rail operation, improves the intelligence of the road-to-rail operation, and improves the conversion efficiency.

[0083] The road-to-rail operating device provided in the present application is described below. The road-to-rail operating device described below and the control method of the road-rail combined transport vehicle described above can be referenced to each other.

[0084] Figure 6 Schematic diagram of the structure of the control device of the road-rail combined transport vehicle provided in the embodiment of the present application. Figure 6 In the control device of the road-rail transport vehicle provided in the embodiment of the present application, the road-rail transport vehicle includes a 3D camera and a plurality of railway steel wheels, the 3D camera is arranged at the bottom of the road-rail transport vehicle, the field of view of the 3D camera faces the ground and covers a part of one of the railway steel wheels; the device includes:

[0085] An acquisition module 610 is used to acquire a depth image captured by the 3D camera after the road-rail transport vehicle travels to a preset area of ​​a railway station;

[0086] A determination module 620 is used to determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway wheel in the image coordinate system;

[0087] The operation module 630 is used to control the road-rail combined transport vehicle to perform road-to-rail operation based on the first straight line equation and the second straight line equation.

[0088] The control device of the road-rail combined transport vehicle provided in the embodiment of the present application obtains a depth image taken by a 3D camera and determines the straight line equations corresponding to the railway rails and the railway wheels in the depth image. Then, it can intelligently guide the road-to-rail operation process of the road-rail combined transport vehicle according to the two straight line equations, thereby realizing the road-to-rail operation of the road-rail combined transport vehicle in an unmanned command mode, reducing the work intensity of personnel, reducing the labor cost of the road-to-rail operation, and improving the conversion efficiency. It can give full play to the advantages of road-rail combined transport and promote the development of the logistics industry.

[0089] Specifically, the above-mentioned road-to-rail operation device provided in the embodiment of the present application can implement all the method steps implemented by the above-mentioned method embodiment in which the execution subject is the controller, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0090] Figure 7 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the control method of the road-rail combined transport vehicle, for example, including:

[0091] After the road-rail transport vehicle travels to a preset area of ​​a railway station, acquiring a depth image captured by the 3D camera;

[0092] Determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system;

[0093] Based on the first straight line equation and the second straight line equation, the road-rail combined transport vehicle is controlled to perform road-to-rail operations.

[0094] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-On l yMemory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0095] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the control method of the road-rail combined transport vehicle provided by the above methods, for example, including:

[0096] After the road-rail transport vehicle travels to a preset area of ​​a railway station, acquiring a depth image captured by the 3D camera;

[0097] Determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system;

[0098] Based on the first straight line equation and the second straight line equation, the road-rail combined transport vehicle is controlled to perform road-to-rail operations.

[0099] In another aspect, the present application further provides a computer program product, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the steps of the control method of the road-rail combined transport vehicle provided by the above methods, for example, including:

[0100] After the road-rail transport vehicle travels to a preset area of ​​a railway station, acquiring a depth image captured by the 3D camera;

[0101] Determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system;

[0102] Based on the first straight line equation and the second straight line equation, the road-rail combined transport vehicle is controlled to perform road-to-rail operations.

[0103] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0105] It should also be noted that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0106] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0107] "Determine B based on A" in the embodiments of the present application means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "Determine B based on A and C", "Determine B based on A, C and E", "Determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "When A meets the first condition, use the first method to determine B"; for another example, "When A meets the second condition, determine B", etc.; for another example, "When A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "When A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0108] In the embodiments of the present application, the term "multiple" refers to two or more than two, and other quantifiers are similar. Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application, rather than to limit it; although the present application is described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solution from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for controlling a road-rail combined transport vehicle, characterized in that: The road-rail transport vehicle comprises a 3D camera and a plurality of railway steel wheels, wherein the 3D camera is arranged at the bottom of the road-rail transport vehicle, and the field of view of the 3D camera faces the ground and covers a portion of one of the railway steel wheels; the method comprises: After the road-rail transport vehicle travels to a preset area of ​​a railway station, acquiring a depth image captured by the 3D camera; Determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system; Based on the first straight line equation and the second straight line equation, the road-rail combined transport vehicle is controlled to perform road-to-rail operations.

2. The control method of the road-rail combined transport vehicle according to claim 1, characterized in that: The controlling the road-rail combined transport vehicle to perform a road-to-rail operation based on the first straight line equation and the second straight line equation includes: Determine an angle between the first straight line and the second straight line based on the first straight line equation and the second straight line equation; Based on the size of the angle, the road-rail combined transport vehicle is controlled to perform road-to-rail operations.

3. The control method of the road-rail combined transport vehicle according to claim 2, characterized in that: The controlling the road-rail combined transport vehicle to perform road-to-rail operation based on the size of the included angle includes: When the size of the angle is within a preset angle range, determining a first distance between the first straight line and the second straight line in the image coordinate system based on the first straight line equation and the second straight line equation; Based on the first spacing, controlling the road-rail combined transport vehicle to complete the road-to-rail operation; or: When the size of the angle is not within the preset angle range, outputting a prompt message to prompt the staff to adjust the position of the road-rail combined transport vehicle so that the size of the angle is within the preset angle range; After the size of the angle is within the preset angle range, re-determining the first straight line equation and the second straight line equation; Determine the first distance between the first straight line and the second straight line in the image coordinate system based on the re-determined first straight line equation and the second straight line equation; Based on the first spacing, the road-rail combined transport vehicle is controlled to complete the road-to-rail operation.

4. The control method of the road-rail combined transport vehicle according to claim 3, characterized in that: The controlling the road-rail combined transport vehicle to complete the road-to-rail operation based on the first spacing includes: Based on the first spacing, determining a second spacing; the second spacing is the actual spacing between the railway rail and the railway steel wheel; Based on the second spacing, the road-rail combined transport vehicle is controlled to complete the road-to-rail operation.

5. The control method of the road-rail combined transport vehicle according to claim 1, characterized in that: The step of determining a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image comprises: Removing the pixel points corresponding to the railway steel wheel in the depth image to obtain a target image; The first straight line equation is determined based on the value of each pixel in the target image.

6. The control method of the road-rail combined transport vehicle according to claim 5, characterized in that: The determining the first straight line equation based on the value of each pixel point in the target image includes: Calculate the difference between the values ​​of two adjacent pixels in each row of the target image to obtain a deviation value; Determine the target pixel based on the size of the deviation value; Perform linear fitting on the target pixel points to obtain the first straight line equation.

7. The control method of the road-rail combined transport vehicle according to claim 1, characterized in that: The step of determining a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system comprises: Determine the coordinate value corresponding to the center line of the railway steel wheel covered by the field of view of the 3D camera in the image coordinate system; Based on the coordinate values, the second straight line equation is determined.

8. A control device for a road-rail combined transport vehicle, characterized in that: The road-rail transport vehicle comprises a 3D camera and a plurality of railway steel wheels, wherein the 3D camera is arranged at the bottom of the road-rail transport vehicle, and the field of view of the 3D camera faces the ground and covers a portion of one of the railway steel wheels; the device comprises: An acquisition module, used for acquiring a depth image taken by the 3D camera after the road-rail combined transport vehicle travels to a preset area of ​​a railway station; A determination module, configured to determine a first straight line equation of a first straight line corresponding to the railway rail in the image coordinate system of the depth image; and determine a second straight line equation of a second straight line corresponding to the railway steel wheel in the image coordinate system; An operation module is used to control the road-rail combined transport vehicle to perform road-to-rail operations based on the first straight line equation and the second straight line equation.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the control method of the road-rail combined transport vehicle as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the control method of the road-rail combined transport vehicle as claimed in any one of claims 1 to 7 is implemented.

Images