Method for controlling telephoto camera cradle head on the basis of track curvature calculation, device, and medium

Abstract

The present invention relates to a method for controlling a telephoto camera cradle head on the basis of track curvature calculation, a device, and a medium. The method comprises: first, on the basis of forward-looking video information obtained by a near-focus camera and a telephoto camera on a train, recognizing, tracking, and calculating a position of a track at a fixed distance ahead; then calculating a track curvature, and further calculating an angle through which the telephoto camera needs to rotate; and finally, on the basis of the calculated angle information, adjusting the field of view of the telephoto camera by means of a cradle head, thereby keeping a forward track area within the field of view of the telephoto camera. Compared with the exiting technology, the present invention has advantages such as improved reliability and safety of detection.

Description

Method, equipment, and medium for telephoto camera gimbal control based on orbit curvature calculation Technical Field

[0001] This invention relates to rail transit signaling systems, and in particular to a method, device, and medium for controlling a telephoto camera gimbal based on track curvature calculation. Background Technology

[0002] With the continuous development of automatic driving technology in rail transit, the use of sensors such as LiDAR, millimeter-wave radar, and cameras for active forward detection of trains has gradually become the mainstream approach to ensuring driving safety. Typically, to ensure identification and detection at both long and short distances, two cameras are used: one with a telephoto lens and the other with a close-up lens. (For ease of explanation, the camera with the telephoto lens will be referred to as a telephoto camera, and the camera with the close-up lens as a close-up camera). However, if a fixed-angle telephoto camera is installed on the train, it often loses its forward field of view due to its small angle of view when the train is turning, causing detection failure and interruption, significantly impacting the detection results.

[0003] There are currently two solutions to this typical problem: (1) limit the focal length to increase the field of view of the telephoto camera, and (2) rotate the camera platform according to the curvature of the track route. Currently, many applications still rely on method 1, which increases the field of view of the telephoto camera by selecting a large image sensor and a restrained focal length, thereby keeping the front view in the picture, or simply relying on the image processing of the close-up camera, with the telephoto image only used as an auxiliary. This method greatly limits the advantage of the telephoto camera in long-distance detection. Therefore, if you want to make full use of the advantages of the telephoto camera, you can only choose method 2. Regarding Method 2, there is currently limited research on camera platform tracking based on track curvature. Some studies use video images or lidar sensors to perceive the shape of the track area ahead, such as patents CN115635993A and CN117622262B, but these are not linked to the telephoto camera platform. Alternatively, positioning information could be obtained based on external positioning information such as GPS or vehicle communication (e.g., patent CN114655276B), and then the shape of the track area could be obtained using a known track model. However, this approach is not suitable for situations where the train may lose its positioning or become inaccurate on the track. Therefore, there is an urgent need for a technical solution that can calculate track curvature in real time and adjust the field of view of the telephoto camera in rail transit. Summary of the Invention

[0004] The purpose of this invention is to overcome the defects of the prior art by providing a telephoto camera gimbal control method, device and medium based on track curvature calculation.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] According to a first aspect of the present invention, a telephoto camera gimbal control method based on track curvature calculation is provided. The method first identifies, tracks and calculates the position of the track at a fixed distance ahead based on the forward-looking video information obtained by the near-focus camera and the telephoto camera on the train. Then, the track curvature is calculated, and the angle that the telephoto camera needs to rotate is further calculated. Finally, the field of view of the telephoto camera is adjusted by the gimbal according to the calculated angle information, so as to keep the track area ahead within the field of view of the telephoto camera.

[0007] As a preferred technical solution, the method specifically includes the following steps:

[0008] Step S1, attitude correction of the close-up and telephoto cameras;

[0009] Step S2: The close-up camera and the telephoto camera acquire real-time images;

[0010] Step S3: Select the lateral detection areas of the near-focus camera and the telephoto camera;

[0011] Step S4: Select the template frame;

[0012] Step S5: Match frame by frame;

[0013] Step S6, calculate the average level offset of the track;

[0014] Step S7, orbital curvature R k calculate;

[0015] Step S8, rotate the telephoto camera by angle J k calculate;

[0016] Step S9, rotate the telephoto camera gimbal to a horizontal angle J k Location.

[0017] As a preferred technical solution, step S1 specifically includes the following steps:

[0018] Step S101: Ensure the train stops on the straight track;

[0019] Step S102: Perform field-of-view calibration for the close-up and telephoto cameras.

[0020] Step S103: Adjust the mounting position of the telephoto camera gimbal to ensure that the horizontal center line of the telephoto camera's field of view remains parallel to the horizontal center line of the near-focus camera's field of view when the gimbal rotates horizontally.

[0021] As a preferred technical solution, in step S2, the near-focus camera and the far-focus camera capture video data in front of the vehicle in real time and transmit it to the industrial control computer.

[0022] As a preferred technical solution, in step S3, the horizontal detection areas A1 and A2 of the track in the video images of the near-focus camera and the far-focus camera are determined, and the horizontal distance between the actual area corresponding to the horizontal detection area and the installation position of the camera is L1 and L2.

[0023] As a preferred technical solution, in step S4, the template frame T of the track in the horizontal detection area of ​​the track in the near-focus camera video frame is determined in the initial frame. 1A and T 1B Subsequently, the best matching box from the previous frame is used instead of the template box; in the initial frame, the template box T of the track in the horizontal detection area of ​​the track in the telephoto camera video frame is determined. 1C and T 1D The best matching box from the previous frame will be used instead of the template box in subsequent frames.

[0024] As a preferred technical solution, in step S4, the template frame Tk of the track is matched frame by frame in the horizontal detection area A1 of the near-focus camera video frame by frame. A Output the best matching box S kA Horizontal pixel distance d kA ;Template box Tk for matching tracks B Output the best matching box S kB Horizontal pixel distance d kB ;

[0025] Match the template frame Tk of the track in the horizontal detection area A2 frame by frame in the video footage from the telephoto camera. C Output the best matching box S kC Horizontal pixel distance d kC ;Template box Tk for matching tracks D Output the best matching box S kD Horizontal pixel distance d kD .

[0026] As a preferred technical solution, the calculation process in step S6 is as follows:

[0027] The horizontal offset distance dmiMdle_k_close of the track center in the kth frame of the near-focus camera is calculated using the following formula:

[0028] The horizontal offset distance dmiddle_k_far of the orbit center in the k-th frame of the telephoto camera is calculated using the following formula:

[0029] The field of view (FOV) of the near-focus camera is (F... H1 F W1 The focal length is f1, and the sensor size is (C). H1 C W1The field of view (FOV) of a telephoto camera is (F... H2 F W2 The focal length is f2, and the sensor size is (C). H2 C W2 ).

[0030] As a preferred technical solution, the calculation process in step S7 is as follows:

[0031] Calculate the radius of curvature R of the track in the kth frame of the near-focus camera. k_close The calculation formula is:

[0032] Calculate the radius of curvature R of the orbit in the kth frame of the telephoto camera. k_far The calculation formula is:

[0033] If the field of view of the telephoto camera is not obstructed, the radius of curvature of the track is calculated based on the telephoto camera's result:

[0034] R k =R k_far

[0035] If the field of view of the telephoto camera is obstructed, then the calculation results of the close-focus camera need to be used:

[0036] R k =R k_close

[0037] .

[0038] As a preferred technical solution, the calculation process in step S8 is as follows:

[0039] Where J0 is the horizontal rotation angle of the telephoto camera's gimbal during attitude correction in step S1.

[0040] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0041] According to a third aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described thereon.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] 1) This invention uses a camera to detect and track the position of the track at a fixed distance in front, thereby calculating and adjusting the rotation angle of the telephoto camera; keeping the track area in front always within the field of view of the telephoto camera, so that subsequent track area and obstacle detection are no longer troubled by the loss of field of view, thus improving the reliability and safety of detection.

[0044] 2) This invention does not rely on external positioning and can be used even when positioning is lost, thus improving the continuity and applicability of video detection;

[0045] 3) This invention increases the upper limit of the usable focal length and increases the size of distant objects in the image, making them clearer and easier to identify, thereby improving the accuracy of visual detection. Attached Figure Description

[0046] Figure 1 is a detailed flowchart of the method of the present invention;

[0047] Figure 2 shows the installation positions of the near-focus and telephoto cameras and their field of view.

[0048] Figure 3 is a schematic diagram of the horizontal detection area and template frame in the field of view of the near-focus camera;

[0049] Figure 4 is a schematic diagram of the horizontal detection area and template frame in the field of view of the telephoto camera. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0051] This invention uses forward-looking video information obtained from close-up and telephoto cameras on a train to identify, track, and calculate the position of the track at a fixed distance ahead, calculate the track curvature, and thus calculate the angle that the telephoto camera needs to rotate. Finally, the telephoto camera's field of view is adjusted by a gimbal to keep the track area ahead within the telephoto camera's field of view.

[0052] As shown in Figure 1, this invention first performs attitude correction on the near-focus camera and the telephoto camera; then, it acquires the video streams of the near-focus camera and the telephoto camera respectively, and begins frame-by-frame processing; a horizontal detection area is selected, and the matching template box for each frame is determined. Matching is performed within the horizontal detection area to obtain the best matching box; the horizontal lateral offset of the track within the horizontal detection area is calculated. Here, the pixel offset is calculated first, and then converted into the actual offset; then, the track curvature is calculated in the near-focus camera processing program to estimate the rotation angle of the telephoto camera. The rotation angle is directly estimated in the telephoto camera processing program. If the field of view of the telephoto camera is not obstructed, a reliable rotation angle can be calculated, and it is used. If a reliable rotation angle cannot be calculated, the rotation angle estimated by the near-focus camera is used. If the near-focus camera is also obstructed and cannot be calculated, the rotation angle remains unchanged.

[0053] The present invention specifically includes the following steps:

[0054] Step S001, Attitude Correction of Close-Focus and Telephoto Cameras: First, ensure the train is stopped on a straight track. Then, calibrate the fields of view of the close-focus and telephoto cameras. Adjust the mounting position of the telephoto camera's gimbal to align the center lines of their fields of view. Ensure that the horizontal center line of the telephoto camera's field of view remains parallel to the horizontal center line of the close-focus camera's field of view when the gimbal rotates horizontally. Record the horizontal rotation angle of the gimbal at this time as J0 (unit: radians), and the field of view (FOV) of the close-focus camera as (F... H1 F W1 The focal length is f1 (in meters), and the sensor size is (C). H1 C W1 (Unit: m), the field of view (FOV) of a telephoto camera is (F... H2 F W2 The focal length is f2 (in meters), and the sensor size is (C). H2 C W2 (Unit: m), where F H1 F W1 These are the horizontal and vertical field of view, C. H1 C W1 These are the horizontal and vertical dimensions, respectively.

[0055] Step S002: Acquire real-time images: The close-up and telephoto cameras capture video data in front of the vehicle in real time and transmit it to the industrial control computer.

[0056] Step S003: Select the horizontal detection area: Determine the horizontal detection areas A1 and A2 of the track in the video images of the near-focus camera and the far-focus camera. The horizontal distance between the actual area corresponding to the horizontal detection area and the installation position of the camera is L1 and L2.

[0057] Step S004: Select template frame: In the initial frame, determine the template frame T of the track in the horizontal detection area of ​​the track in the close-up camera video frame.1A and T 1B The best matching box from the previous frame can be used to replace the template box in subsequent frames; the template box T of the track in the horizontal detection area of ​​the track in the telephoto camera video frame is determined in the initial frame. 1C and T 1D The best matching box from the previous frame can be used to replace the template box in subsequent frames.

[0058] Step S005, Frame-by-frame matching: In the close-up camera video frame by frame (assuming the k-th frame), match the template box Tk of the track in the horizontal detection area A1. A Output the best matching box S kA Horizontal pixel distance d kA ;Template box Tk for matching tracks B Output the best matching box S kB Horizontal pixel distance d kB In the video footage from the telephoto camera, match the template frame Tk of the track in the horizontal detection area A2 frame by frame. C Output the best matching box S kC Horizontal pixel distance d kC ;Template box Tk for matching tracks D Output the best matching box S kD Horizontal pixel distance d kD .

[0059] Step S006, Calculation of the horizontal offset of the track: Calculate the horizontal offset distance dmiddle_k_close (unit: meters) of the track center in the kth frame of the close-focus camera. The calculation formula is as follows:

[0060] Calculate the horizontal offset distance dmiddle_k_far (in meters) of the orbit center of the telephoto camera in the kth frame. The formula is:

[0061] Step S007, Track Curvature Calculation: Calculate the average curvature R of this curve segment. k The calculation formula is as follows:

[0062] Calculate the radius of curvature R of the track in the kth frame of the near-focus camera. k_close The calculation formula is:

[0063] Calculate the radius of curvature R of the orbit in the kth frame of the telephoto camera. k_far The calculation formula is:

[0064] If the field of view of the telephoto camera is not obstructed, the radius of curvature of the track is calculated based on the telephoto camera's result:

[0065] R k=R k_far

[0066] If the field of view of the telephoto camera is obstructed, then the calculation results of the close-focus camera need to be used:

[0067] R k =R k_close

[0068] Step S008, Calculation of telephoto camera rotation angle: Calculate the required horizontal rotation angle J of the telephoto camera gimbal at this time. k Unit: radians, calculation formula is as follows:

[0069] Step S009, Telephoto Camera Gimbal Rotation: Rotate the telephoto camera gimbal to a horizontal angle J k Location.

[0070] Figure 2 shows the installation positions and field of view of the near and far-focus cameras. The two cameras are stacked, keeping their relative positions unchanged, and the horizontal lines of the cameras are consistent. The field of view of the far-focus camera is part of the field of view of the near-focus camera.

[0071] Figures 3 and 4 are schematic diagrams of the horizontal detection region and template frame in the field of view of near-focus and far-focus cameras, respectively. The horizontal detection region is a box with a fixed position and fixed pixel size. The position is selected empirically and can be chosen at the lower 1 / 3 of the image. The template frame is a box with the same height as the horizontal detection region and a settable length, generally set to 1.5-2.5 times the height. The initial template frame is selected as a part of the horizontal detection region, including the railway track. When performing frame-by-frame matching, only the box with the highest similarity to the template frame in the horizontal detection region needs to be calculated, which is the best matching box.

[0072] The above is an introduction to the method embodiments. The following embodiments using electronic devices and storage media will further illustrate the solution of the present invention.

[0073] This invention also provides an electronic device including a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0074] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] The processing unit executes the various methods and processes described above, such as methods S001 to S009. For example, in some embodiments, methods S001 to S009 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of methods S001 to S009 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S001 to S009 by any other suitable means (e.g., by means of firmware).

[0076] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0077] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0078] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0079] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for controlling a remote camera gimbal based on orbit curvature calculation, characterized in that, The method first obtains the front view video information from the close-up camera and the telephoto camera on the train, identifies, tracks and calculates the position of the track at a fixed distance in front, then calculates the track curvature, further calculates the angle at which the telephoto camera needs to be rotated, and finally adjusts the field of view of the telephoto camera according to the calculated angle information through the pan-tilt head, so as to keep the front track area in the field of view of the telephoto camera.

2. The method of claim 1, wherein the method further comprises: The method specifically comprises the following steps: Step S1, pose correction of the close-up camera and the telephoto camera; Step S2, the close-up camera and the telephoto camera acquire real-time images; Step S3, selection of the lateral detection area of the close-up camera and the telephoto camera; Step S4, selection of the template frame; Step S5, frame-by-frame matching; Step S6, average horizontal offset calculation of the track; Step S7, track curvature R k Calculation; Step S8, rotation angle J of tele camera k calculations; Step S9, rotate the telephoto camera holder to a horizontal angle J k Position.

3. The method of claim 2, wherein the control method is based on the curvature of the orbit. The step S1 specifically comprises the following steps: Step S101, ensure that the train is stopped on a straight track; Step S102, perform field of view calibration of the close-up camera and the telephoto camera Step S103, adjust the installation pose of the pan-tilt head of the telephoto camera to ensure that the horizontal midline of the field of view of the telephoto camera is parallel to the horizontal midline of the field of view of the close-up camera when the pan-tilt head is horizontally rotated.

4. The control method of claim 2, wherein, In the step S2, the close-up camera and the telephoto camera capture real-time video data in front of the vehicle running and transmit the data to the industrial computer.

5. The method of claim 2, wherein the method further comprises: In the step S3, the lateral detection areas A1 and A2 of the track in the video pictures of the close-up camera and the telephoto camera are determined, and the horizontal distances L1 and L2 between the actual areas corresponding to the lateral detection areas and the installation positions of the cameras are determined.

6. The method of claim 5, wherein, In said step S4, a template frame T of the track in the lateral detection area of the track in the near-focus camera video picture is determined in the initial frame 1A and T 1B , the best matching frame of the previous frame is used instead of the template frame in the subsequent frames. determining a template box T of the track in the lateral detection area of the track in the tele camera video picture in the initial frame 1C and T 1D , the best matching box of the previous frame is used instead of the template box in the subsequent.

7. The method of claim 6, wherein the method further comprises: Said step S4, in the near-focus camera video frame in the lateral detection area A1 in the matching track template frame Tk A , output the horizontal pixel distance d kA of the best matching frame S kA ; Match the track template frame Tk B , output the horizontal pixel distance d kB of the best matching frame S kB ; In the frame-by-frame matching of the template frame Tk of the track in the lateral detection area A2 in the video picture of the fixed-focus camera C , output the horizontal pixel distance d of the best matching frame S kC ; match the template frame Tk of the track, output the horizontal pixel distance d of the best matching frame S kC . D kD kD .​​ 8. The method of claim 6, wherein the method further comprises: In the step S6, the calculation process is as follows: The horizontal offset distance dmiddle_k_close of the track center of the kth frame of the close-up camera is calculated according to the following formula: The horizontal offset distance dmiddle_k_far of the track center of the kth frame of the telephoto camera is calculated, and the calculation formula is: where the field of view FOV of the close-up camera is (F H1 , F W1 ), the focal length is f1, and the photosensitive element size is (C H1 , C W1 ), the field of view FOV of the telephoto camera is (F H2 , F W2 ), the focal length is f2, and the photosensitive element size is (C H2 , C W2 ).

9. The method of claim 8, wherein, In the step S7, the calculation process is as follows: calculating a radius of curvature R of the kth frame track of the close-up camera k_close , the calculation formula is: calculating a radius of curvature R of the kth track of the telecentric camera k_far , the calculation formula is: If the field of view of the telephoto camera is not blocked, the curvature radius of the track is calculated according to the calculation result of the telephoto camera: R k = R k_far If the field of view of the telephoto camera is blocked, the calculation result of the close-up camera needs to be used: R k = R k_close 。 10. The method of claim 8, wherein, The calculation process in step S8 is as follows: Wherein J0 is the horizontal rotation angle of the pan-tilt head of the telephoto camera in the pose correction in the step S1.

11. An electronic device comprising a memory and a processor, said memory having stored thereon a computer program, characterized in that, The processor executes the program to implement the method in any one of claims 1-10.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method in any one of claims 1-10.

Images