Track curvature calculation-based far-focus camera holder control method, equipment and medium

Abstract

The invention relates to a far-focus camera pan-tilt control method and device based on track curvature calculation and a medium, and the method comprises the steps: firstly, based on foresight video information obtained by a near-focus camera and a far-focus camera on a train, recognizing, tracking and calculating the position of a track at a fixed distance in front, then calculating the track curvature, and finally calculating the track curvature; and finally, the view field of the far-focus camera is adjusted through the holder according to the calculated angle information, so that the front track area is kept in the view field of the far-focus camera. Compared with the prior art, the method has the advantages that the detection reliability and safety are improved, and the like.

Description

Technical Field

[0001] The present invention relates to a rail transit signal system, and in particular to a telephoto camera pan / tilt control method, device and medium based on track curvature calculation. Background Art

[0002] With the continuous development of rail transit autonomous driving technology, the use of sensors such as lidar, millimeter-wave radar, and cameras for active forward detection of trains has gradually become the mainstream to ensure driving safety. In order to ensure identification and detection at long and short distances, two cameras are usually used, one with a telephoto lens and the other with a close-focus lens. (For the convenience of description, the camera with a telephoto lens will be referred to as a telephoto camera, and the camera with a close-focus lens will be referred to as a close-focus camera.) However, if a fixed-angle telephoto camera is installed on the train, then when the train is turning, the telephoto camera will often lose its view of the road ahead due to its small field of view, causing failure and interruption of detection, which greatly affects the detection effect.

[0003] There are currently two solutions to this typical problem: (1) limit the focal length and increase the telephoto camera's field of view, and (2) rotate the camera platform according to the curvature of the track route. Currently, many applications still use method 1, which relies on selecting large photosensitive elements and restraining the focal length to increase the telephoto camera's field of view, thereby keeping the front view in the picture, or simply relying on the image processing of the near-focus camera, with the telephoto picture only as an auxiliary. This method greatly limits the advantages of the telephoto camera in long-distance detection. Therefore, if you want to fully utilize the advantages of the telephoto camera, you can only choose method 2. As for method 2, there are currently few studies on camera platform tracking based on the curvature of the track route. Some studies will perceive the shape of the track area ahead through video images or lidar sensors, such as patents CN115635993A and CN117622262B, but they are not linked to the telephoto camera platform; perhaps positioning information can be obtained through external positioning information such as GPS and on-board communications (such as patent CN114655276B), and then the track area shape can be obtained through a known track model, but it is not applicable to situations where the train may lose its position or be inaccurately positioned on the track. Therefore, there is an urgent need for a technical solution that can calculate the track curvature in real time and adjust the field of view of the telephoto camera in rail transit. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a telephoto camera gimbal control method, device and medium based on track curvature calculation.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to a first aspect of the present invention, a method for controlling a telephoto camera gimbal based on track curvature calculation is provided. The method first identifies, tracks and calculates the position of the track at a fixed distance ahead based on the forward-looking video information obtained by the near-focus camera and the telephoto camera on the train, then calculates the track curvature, and further calculates the angle at which the telephoto camera needs to rotate. Finally, the field of view of the telephoto camera is adjusted by the gimbal according to the calculated angle information, thereby keeping the front track area within the field of view of the telephoto camera.

[0007] As a preferred technical solution, the method specifically comprises the following steps:

[0008] Step S1, posture correction of the near-focus camera and the far-focus camera;

[0009] Step S2, the near-focus camera and the far-focus camera acquire real-time images;

[0010] Step S3, selecting the lateral detection areas of the near-focus camera and the far-focus camera;

[0011] Step S4, selecting a template frame;

[0012] Step S5, frame-by-frame matching;

[0013] Step S6, calculating the average horizontal deviation of the track;

[0014] Step S7, track curvature R k calculate;

[0015] Step S8: The telephoto camera rotates to an angle J k calculate;

[0016] Step S9, rotate the telephoto camera gimbal to a horizontal angle J k Location.

[0017] As a preferred technical solution, step S1 specifically includes the following steps:

[0018] Step S101, ensuring that the train stops on a straight track;

[0019] Step S102: Calibrate the near-focus camera and the far-focus camera field of view

[0020] Step S103, adjusting the installation posture of the telephoto camera gimbal to ensure that the horizontal center line of the telephoto camera's field of view remains parallel to the horizontal center line of the near-focus camera's field of view when the gimbal rotates horizontally.

[0021] As a preferred technical solution, in step S2, the near-focus camera and the far-focus camera capture the video data in front of the vehicle in real time and transmit it to the industrial computer.

[0022] As a preferred technical solution, in step S3, the lateral detection areas A1 and A2 of the track in the video images of the near-focus camera and the far-focus camera are determined, and the horizontal distances between the actual areas corresponding to the lateral detection areas and the installation positions of the cameras are L1 and L2.

[0023] As a preferred technical solution, in step S4, the template frame T of the track in the horizontal detection area of ​​the track in the video picture of the close-focus camera is determined in the initial frame. 1A and T 1B , and then use the best matching frame of the previous frame to replace the template frame; determine the template frame T of the track in the horizontal detection area of ​​the track in the video picture of the telephoto camera in the initial frame 1C and T 1D , and subsequently use the best matching box of the previous frame to replace the template box.

[0024] As a preferred technical solution, in step S4, the template frame Tk of the track is matched frame by frame in the horizontal detection area A1 in the close-focus camera video picture. A , output the best matching box S kA The horizontal pixel distance d kA ;Template box Tk matching track B , output the best matching box S kB The horizontal pixel distance d kB ;

[0025] Match the track template box Tk in the horizontal detection area A2 frame by frame in the telephoto camera video frame C , output the best matching box S kC The horizontal pixel distance d kC ;Template box Tk matching track D , output the best matching box S kD The horizontal pixel distance d kD .

[0026] As a preferred technical solution, the calculation process in step S6 is as follows:

[0027] Calculate the horizontal offset distance of the center of the k-th frame track of the close-focus camera dmiddle_k_close , the calculation formula is:

[0028]

[0029] Calculate the horizontal offset distance d of the telephoto camera's track center in frame k middle_k_far , the calculation formula is:

[0030]

[0031] The field of view FOV of the close-focus camera is (F H1 , F W1), the focal length is f1, and the sensor size is (C H1 , C W1 ), the field of view FOV of the telephoto camera is (F H2 , F W2 ), the focal length is f2, and the sensor size is (C H2 , C w2 ).

[0032] As a preferred technical solution, the calculation process in step S7 is as follows:

[0033] Calculate the curvature radius R of the k-th frame track of the close-focus camera k_close , the calculation formula is:

[0034]

[0035] Calculate the radius of curvature R of the telephoto camera track at frame k k_far , the calculation formula is:

[0036]

[0037] If the telephoto camera's field of view is not blocked, the radius of curvature of the track is calculated according to the telephoto camera:

[0038] R k =R k_far

[0039] If the far-focus camera's field of view is blocked, the calculation results of the near-focus camera need to be used:

[0040] R k =R k_close .

[0041] As a preferred technical solution, the calculation process in step S8 is as follows:

[0042]

[0043] Wherein J0 is the horizontal rotation angle of the gimbal of the telephoto camera during the posture correction in step S1.

[0044] According to a second aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0045] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method described above when executed by a processor.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] 1) The present invention detects, tracks and calculates the position of the track at a fixed distance in front through a camera, thereby calculating and adjusting the rotation angle of the telephoto camera; the front track area is kept within the field of view of the telephoto camera, so that subsequent track area and obstacle detection are no longer troubled by the loss of field of view, thereby improving the reliability and safety of detection;

[0048] 2) The present invention does not need to rely on external positioning and can be used even when positioning is lost, thereby improving the continuity and applicability of video detection;

[0049] 3) The present invention increases the upper limit of the usable focal length and increases the size of distant objects in the picture, making them clearer and easier to identify, thereby improving the accuracy of visual detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a specific flow chart of the method of the present invention;

[0051] Figure 2 This is a schematic diagram of the installation positions of the near-focus and far-focus cameras and their field of view;

[0052] Figure 3 It is a schematic diagram of the horizontal detection area and template frame in the field of view of the close-focus camera;

[0053] Figure 4 Schematic diagram of the lateral detection area and template frame in the field of view of the telephoto camera. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0055] The present invention is based on the forward-looking video information obtained by the near-focus camera and the far-focus camera on the train, identifies, tracks and calculates the position of the track at a fixed distance ahead, calculates the track curvature, and thus calculates the angle that the far-focus camera needs to rotate. Finally, the field of view of the far-focus camera is adjusted through the pan / tilt system, thereby achieving the purpose of keeping the front track area within the field of view of the far-focus camera.

[0056] like Figure 1As shown, the present invention first performs posture correction of the near-focus camera and the far-focus camera; then obtains the video streams of the near-focus camera and the far-focus camera respectively, and starts frame-by-frame processing; selects a horizontal detection area, determines the matching template frame of each frame, and performs matching in the horizontal detection area to obtain the best matching frame; calculates the horizontal lateral offset of the track in the horizontal detection area, where the pixel offset is first calculated and then converted into the actual offset; then, calculates the track curvature in the near-focus camera processing program to estimate the rotation angle of the far-focus camera, and directly estimates the rotation angle in the far-focus camera processing program. If the field of view of the far-focus camera is not blocked, a reliable rotation angle can be calculated and adopted. If the reliable rotation angle cannot be calculated, the rotation angle inferred by the near-focus camera is adopted. If the near-focus camera is also blocked and cannot be calculated, the rotation angle is kept unchanged.

[0057] The present invention specifically comprises the following steps:

[0058] Step S001, posture calibration of near-focus camera and far-focus camera: first ensure that the train stops on a straight track, then calibrate the near-focus camera and far-focus camera field of view, adjust the installation posture of the far-focus camera gimbal, align the center lines of the two fields of view, ensure that the horizontal center line of the far-focus camera field of view is parallel to the horizontal center line of the near-focus camera field of view when the gimbal rotates horizontally, record the horizontal rotation angle of the gimbal at this time as J0 (unit: radian), and the field of view angle FOV of the near-focus camera is (F H1 , F W1 ), focal length is f1 (unit: m), photosensitive element size is (C H1 , C W1 )(unit: m), the field of view FOV of the telephoto camera is (F H2 , F W2 ), focal length is f2 (unit: m), photosensitive element size is (C H2 , C W2 )(unit: m), where F H1 、F W1 are the horizontal and vertical field of view, C H1 , C W1 Horizontal and vertical dimensions respectively.

[0059] Step S002, obtaining real-time images: the near-focus camera and the far-focus camera capture the video data in front of the vehicle in real time and transmit it to the industrial computer.

[0060] Step S003, selecting a lateral detection area: determining lateral detection areas A1 and A2 of the track in the video images of the near-focus camera and the far-focus camera, and the horizontal distance between the actual area corresponding to the lateral detection area and the installation position of the camera is L1 and L2.

[0061] Step S004, selecting a template frame: determining the template frame T of the track in the horizontal detection area of ​​the track in the video image of the close-focus camera in the initial frame.1A and T 1B , the best matching frame of the previous frame can be used to replace the template frame later; in the initial frame, the template frame T of the track in the horizontal detection area of ​​the track in the video picture of the telephoto camera is determined 1C and T 1D , the best matching box of the previous frame can be used to replace the template box later.

[0062] Step S005, frame-by-frame matching: match the track template frame Tk in the horizontal detection area A1 frame by frame (assuming the kth frame) in the close-focus camera video image A , output the best matching box S kA The horizontal pixel distance d kA ;Template box Tk matching track B , output the best matching box Sk B The horizontal pixel distance dk B ; Match the track template frame Tk in the horizontal detection area A2 frame by frame in the telephoto camera video frame C , output the best matching box S kC The horizontal pixel distance d kC ;Template box Tk matching track D , output the best matching box S kD The horizontal pixel distance d kD .

[0063] Step S006, horizontal track offset calculation: calculate the horizontal offset distance d of the track center of the kth frame of the close-focus camera middle_k_close (Unit: meter), calculation formula:

[0064]

[0065] Calculate the horizontal offset distance d of the telephoto camera's track center in frame k middle_k_far (Unit: meter), calculation formula:

[0066]

[0067] Step S007, track curvature calculation: calculate the average curvature Rk of the curve segment, the calculation formula is as follows:

[0068] Calculate the curvature radius R of the k-th frame track of the close-focus camera k_close , the calculation formula is:

[0069]

[0070] Calculate the radius of curvature R of the telephoto camera track at frame k k_far , the calculation formula is:

[0071]

[0072] If the telephoto camera's field of view is not blocked, the radius of curvature of the track is calculated according to the telephoto camera:

[0073] R k =R k_far

[0074] If the far-focus camera's field of view is blocked, the calculation results of the near-focus camera need to be used:

[0075] R k =R k_close

[0076] Step S008, calculation of the telephoto camera rotation angle: reverse calculation of the horizontal rotation angle Jk required by the telephoto camera gimbal at this time, unit: radian, the calculation formula is as follows:

[0077]

[0078] Step S009, rotating the telephoto camera platform: rotating the telephoto camera platform to a horizontal angle Jk position.

[0079] Figure 2 This is a schematic diagram of the installation position of the near-focus and far-focus cameras and their field of view. The two cameras are stacked to keep their relative positions unchanged and the horizontal lines of the cameras consistent. The field of view of the far-focus camera is part of the field of view of the near-focus camera.

[0080] Figure 3 and Figure 4 The following are schematic diagrams of the lateral detection area and template frame in the field of view of the near-focus and far-focus cameras, respectively. The lateral detection area is a square frame with a fixed position and fixed pixel size. The position is selected based on experience and can be selected in the lower 1 / 3 of the image. The template frame is a square frame with the same height as the lateral detection area and an adjustable length, which is generally set to 1.5-2.5 times the height. The initial template frame is selected as a part of the lateral detection area, including the rails. When performing frame-by-frame matching, it is only necessary to calculate the square frame with the greatest similarity to the template frame in the lateral detection area, which is the best matching frame.

[0081] The above is an introduction to the method embodiment. The following is a further explanation of the scheme of the present invention through electronic equipment and storage medium embodiments.

[0082] The embodiment of the present invention also provides an electronic device including a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or computer program instructions loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0083] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0084] The processing unit performs the various methods and processes described above, such as methods S001 to S009. For example, in some embodiments, methods S001 to S009 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S001 to S009 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S001 to S009 in any other appropriate manner (e.g., by means of firmware).

[0085] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0086] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0087] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0088] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A telephoto camera gimbal control method based on track curvature calculation, characterized in that: This method first identifies, tracks and calculates the position of the track at a fixed distance ahead based on the forward-looking video information obtained by the near-focus camera and the far-focus camera on the train. It then calculates the track curvature and further calculates the angle at which the far-focus camera needs to rotate. Finally, the field of view of the far-focus camera is adjusted through the gimbal according to the calculated angle information, thereby keeping the front track area within the field of view of the far-focus camera.

2. The telephoto camera gimbal control method based on track curvature calculation according to claim 1, characterized in that: The method specifically comprises the following steps: Step S1, posture correction of the near-focus camera and the far-focus camera; Step S2, the near-focus camera and the far-focus camera acquire real-time images; Step S3, selecting the lateral detection areas of the near-focus camera and the far-focus camera; Step S4, selecting a template frame; Step S5, frame-by-frame matching; Step S6, calculating the average horizontal deviation of the track; Step S7, track curvature R k calculate; Step S8: The telephoto camera rotates to an angle J k calculate; Step S9, rotate the telephoto camera gimbal to a horizontal angle J k Location.

3. The telephoto camera gimbal control method based on track curvature calculation according to claim 2, characterized in that: The step S1 specifically includes the following steps: Step S101, ensuring that the train stops on a straight track; Step S102: Calibrate the near-focus camera and the far-focus camera field of view Step S103, adjusting the installation posture of the telephoto camera gimbal to ensure that the horizontal center line of the telephoto camera's field of view remains parallel to the horizontal center line of the near-focus camera's field of view when the gimbal rotates horizontally.

4. The telephoto camera gimbal control method based on track curvature calculation according to claim 2, characterized in that: In step S2, the near-focus camera and the far-focus camera capture the video data in front of the vehicle in real time and transmit it to the industrial computer.

5. The telephoto camera gimbal control method based on track curvature calculation according to claim 2, characterized in that: In step S3, the lateral detection areas A1 and A2 of the track in the video images of the near-focus camera and the far-focus camera are determined, and the horizontal distances between the actual areas corresponding to the lateral detection areas and the installation positions of the cameras are L1 and L2.

6. The telephoto camera gimbal control method based on track curvature calculation according to claim 5, characterized in that: In step S4, a template frame T of the track in the horizontal detection area of ​​the track in the video picture of the close-focus camera is determined in the initial frame. 1A and T 1B , and then use the best matching box of the previous frame to replace the template box; Determine the template frame T of the track in the lateral detection area of ​​the track in the video frame of the telephoto camera in the initial frame 1C and T 1D , and subsequently use the best matching box of the previous frame to replace the template box.

7. The telephoto camera gimbal control method based on track curvature calculation according to claim 6, characterized in that: In step S4, the template frame Tk of the track is matched frame by frame in the horizontal detection area A1 in the close-focus camera video picture. A , output the best matching box S kA The horizontal pixel distance d kA ;Template box Tk matching track B , output the best matching box S kB The horizontal pixel distance d kB ; Match the track template box Tk in the horizontal detection area A2 frame by frame in the telephoto camera video frame C , output the best matching box S kC The horizontal pixel distance d kC ;Template box Tk matching track D , output the best matching box S kD The horizontal pixel distance d kD .

8. The telephoto camera gimbal control method based on track curvature calculation according to claim 6, characterized in that: The calculation process in step S6 is as follows: Calculate the horizontal offset distance d of the center of the track of the close-focus camera in the kth frame middle_k_close , the calculation formula is: Calculate the horizontal offset distance d of the telephoto camera's track center in frame k middle_k_far , the calculation formula is: The field of view FOV of the close-focus camera is (F H1 , F W1 ), the focal length is f1, and the sensor size is (C H1 , C W1 ), the field of view FOV of the telephoto camera is (F H2 , F W2 ), the focal length is f2, and the sensor size is (C H2 , C W2 ).

9. The telephoto camera gimbal control method based on track curvature calculation according to claim 8, characterized in that: The calculation process in step S7 is as follows: Calculate the curvature radius R of the k-th frame track of the close-focus camera k_close , the calculation formula is: Calculate the radius of curvature R of the telephoto camera track at frame k k_far , the calculation formula is: If the telephoto camera's field of view is not blocked, the radius of curvature of the track is calculated according to the telephoto camera: R k =R k_far If the far-focus camera's field of view is blocked, the calculation results of the near-focus camera need to be used: R k =R k_close 。 10. The telephoto camera gimbal control method based on track curvature calculation according to claim 8, characterized in that: The calculation process in step S8 is as follows: Wherein J0 is the horizontal rotation angle of the gimbal of the telephoto camera during the posture correction in step S1.

11. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images