A track displacement detection system, method and camera

Abstract

The application discloses a track displacement detection system, after starting, whether the vertical drive VD information of the wide dynamic CMOS sensor is updated, after the VD information is updated, the infrared lamp is turned on: the super depth of field camera is used for aiming at the reflecting target, collecting the optical signal and transmitting the optical signal to the wide dynamic CMOS sensor; the wide dynamic CMOS sensor is used for acquiring a dark frame image under regional exposure and a bright frame image under global exposure, transmitting the dark frame image to a first ISP and transmitting the bright frame image to a second ISP; the first ISP is used for correcting the gamma curve of the dark frame image in a manner of increasing contrast, when the corrected dark frame image data represents that the train track does not move, storing current exposure parameters, the current exposure parameters including a current brightness expectation, an actual brightness and an effective exposure amount; when the corrected dark frame image data represents that the train track moves, generating alarm information; the second ISP is used for acquiring the bright frame image as monitoring data.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a track displacement detection system, method and camera. Background Technology

[0002] Due to the immense weight and weight of trains, rapid movement can cause track shifts in some areas due to gravity, inertia, and other factors. Track shifts can lead to derailments, collisions, and other safety accidents. Therefore, timely detection of track movement is crucial. Currently, track movement is typically determined manually at regular intervals. Manual surveying is inefficient, while camera surveying requires extremely high-resolution images to clearly show track movement. Furthermore, train tracks are long, and in areas far from the station, there is often no electrical power supply. Using camera lighting would require large battery capacity, and installing streetlights would significantly increase energy consumption and hardware costs. Therefore, finding a more energy-efficient and effective way to improve track monitoring, and to achieve more efficient, accurate, low-cost, and automated track safety detection, is a pressing issue. Summary of the Invention

[0003] The purpose of this application is to provide a track displacement detection system, method, and camera to solve the problem that train track safety inspection relies on manual surveying.

[0004] In a first aspect, this application provides a track displacement detection system, including a camera and a reflective target located on a train track. The camera includes: an infrared lamp, a fixed-focus, fixed-aperture ultra-depth-of-field camera, a wide dynamic range CMOS sensor, a first ISP, and a second ISP.

[0005] After the camera is started, it detects whether the vertical drive VD information of the wide dynamic range CMOS sensor has been updated. If the VD information is updated, the infrared light is turned on.

[0006] The ultra-depth-of-field camera is used to aim at the reflective target and collect optical signals to transmit to the wide dynamic range CMOS sensor.

[0007] The wide dynamic range CMOS sensor is used to acquire dark frame images under regional exposure and bright frame images under global exposure, and transmits the dark frame images to the first ISP and the bright frame images to the second ISP.

[0008] The first ISP is used to correct the gamma curve of the dark frame image in a way that increases contrast: when the corrected dark frame image data indicates that the train track has not moved, it stores the current exposure parameters, which include the current brightness expectation, actual brightness, and effective exposure; when the corrected dark frame image data indicates that the train track has moved, it generates an alarm message.

[0009] The second ISP is used to acquire bright frame images as monitoring data.

[0010] Optionally, during the process of correcting the gamma curve of the dark frame image in a manner that increases contrast, the first ISP is further configured to:

[0011] Brighten pixels in the dark frame image that have a brightness greater than Y; and / or

[0012] In the dark frame image, pixels with brightness lower than Y-(JL) are darkened, where Y represents the preset desired brightness, J represents the maximum depth of field of the ultra-depth-of-field camera, and L represents the distance between the camera and the reflective target; and / or

[0013] Set the brightness value of pixels in the dark frame image whose brightness is lower than twice the black level brightness to zero.

[0014] Optionally, the reflective target includes two circular areas made of reflective material, wherein the reflective material includes infrared reflective pigment and / or aluminum;

[0015] The first ISP is further configured to detect two circular regions in the dark frame image by means of at least one of Hough Transform, Contour-Based Detection, RANSAC circle fitting algorithm or deep learning after correcting the gamma curve of the dark frame image in a manner that increases contrast.

[0016] The center positions of two circular regions can be detected by at least one of the following methods: perpendicular bisector method, folding method, and right angle method.

[0017] Optionally, the first ISP is further configured to:

[0018] Obtain the distance between the centers of two circular regions;

[0019] When the center distance between the two circular regions meets the preset center distance, determine whether the center positions of the two circular regions are offset:

[0020] If the center positions of the two circular areas are offset, it is determined that the train track has moved, an alarm message is generated, prompting manual verification of whether the train track has moved, and the second ISP is notified to store the bright frame image within a preset time period.

[0021] If the center positions of the two circular areas do not shift, it is determined that the train track has not moved. After storing the current exposure parameters, the camera is instructed to go into sleep mode, and the infrared lights are instructed to turn off.

[0022] Since the camera only starts detecting whether the center of a circular area is off when the center distance between the two circular areas meets the preset center distance, it can avoid misjudgment caused by ordinary reflective points such as garbage and glass, and avoid calculating the center position of each circular area, thus avoiding wasting computing resources.

[0023] Optionally, the first or second ISP may also be used for:

[0024] The camera is activated at a preset detection time, including nighttime hours, by heartbeat monitoring. The vertical drive VD information of the wide dynamic range CMOS sensor is updated by the first ISP and / or the second ISP. After the VD information is updated, the infrared light is turned on, and the wide dynamic range CMOS sensor is controlled to perform exposure based on the historical expected brightness, actual brightness, and effective exposure.

[0025] Optionally, the camera further includes a battery, and the first or second ISP is further used for:

[0026] When the actual brightness steadily decreases in the historical records, the effective exposure amount is increased;

[0027] Based on the trend of changes in effective exposure at nighttime in the historical records, determine whether the battery is degrading;

[0028] After determining that the effective exposure is slowly increasing, the battery is judged to be degrading, and the camera's location information and battery status are sent to the user as a prompt message.

[0029] Optionally, the camera further includes a battery, and the first or second ISP is also used for;

[0030] When the actual brightness steadily decreases in the historical records, the effective exposure amount is increased;

[0031] Based on the trend of changes in effective exposure at nighttime in the historical records, determine whether the camera is dirty;

[0032] After confirming that the effective exposure has stabilized after a sharp increase, and determining that the camera is dirty, the camera's location information and a bright frame image taken during the day are sent to the user as a notification.

[0033] Optionally, the depth-of-field range of the ultra-depth-of-field camera is 3 to 21 meters, the distance between the reflective target and the camera is 3 to 21 meters, the frame rate of the camera is 12 frames per second, and the infrared light is an LWIR long-wave infrared fill light with a wavelength of 950nm.

[0034] Secondly, embodiments of this application also provide a method for detecting track displacement, including:

[0035] After the camera is activated, the vertical drive (VD) information of the wide dynamic range CMOS sensor is checked via the first ISP and / or the second ISP. If the VD information is updated, the infrared LED is turned on.

[0036] Optical signals are acquired from a reflective target using an ultra-depth-of-field camera and transmitted to a wide dynamic range CMOS sensor, wherein the reflective target is located on a train track.

[0037] The wide dynamic range CMOS sensor acquires dark frame images under regional exposure and bright frame images under global exposure, and transmits the dark frame images to the first ISP and the bright frame images to the second ISP.

[0038] The first ISP corrects the gamma curve of the dark frame image by increasing contrast. When the corrected dark frame image data represents the train track without movement, the current exposure parameters are stored. The current exposure parameters include the current brightness expectation, actual brightness, and effective exposure.

[0039] Bright frame images are acquired as monitoring data through the second ISP.

[0040] Thirdly, embodiments of this application also provide a camera, including:

[0041] Fixed-focus, fixed-aperture ultra-depth-of-field camera, wide dynamic range CMOS sensor, first ISP and second ISP;

[0042] After the camera is started, it detects whether the vertical drive VD information of the wide dynamic range CMOS sensor is updated through the first ISP and / or the second ISP. After the VD information is updated, the infrared light is turned on.

[0043] The ultra-depth-of-field camera is used to aim at a reflective target and collect optical signals to transmit to the wide dynamic range CMOS sensor, wherein the reflective target is located on the train track;

[0044] The wide dynamic range CMOS sensor is used to acquire dark frame images under regional exposure and bright frame images under global exposure, and transmits the dark frame images to the first ISP and the bright frame images to the second ISP.

[0045] The first ISP is used to correct the gamma curve of the dark frame image in a way that increases contrast: when the corrected dark frame image data indicates that the train track has not moved, it stores the current exposure parameters, which include the current brightness expectation, actual brightness, and effective exposure; when the corrected dark frame image data indicates that the train track has moved, it generates an alarm message.

[0046] The second ISP is used to acquire bright frame images as monitoring data.

[0047] In this embodiment, the decision to turn on the infrared lamp is determined by whether the VD information of the wide dynamic range CMOS sensor is updated. This avoids wasting camera energy by keeping the infrared lamp constantly on when there is no outflow. Double exposure is performed using the wide dynamic range CMOS sensor. When the dark frame image data indicates that the train track has not moved, the current exposure parameters are stored, reducing memory consumption. Furthermore, optical signals are acquired using a reflective target, eliminating the need for additional light sources and frequent manual calibration. This significantly improves the shooting effect of the camera near the train track and reduces overall energy consumption. Since judging whether the train track has moved is more accurate through the calibrated dark frame image, repeated manual surveying is unnecessary, greatly improving the efficiency of train track detection. This achieves efficient, low-cost, and automated train track safety detection, preventing accidents caused by train track movement and ensuring safe train travel. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of a reflective target in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram of a reflective target affixed to a train track in an embodiment of this application.

[0050] Figure 3 This is a schematic diagram showing the dimensions of the reflective target in an embodiment of this application;

[0051] Figure 4 This is a schematic diagram of the gamma curve of the dark frame in the embodiments of this application;

[0052] Figure 5 This is a schematic diagram of the corrected reflective target in an embodiment of this application;

[0053] Figure 6 This is a schematic diagram of the gamma curve of the bright frame in the embodiments of this application;

[0054] Figure 7 This is a schematic diagram of the track displacement detection method in the embodiments of this application;

[0055] Figure 8 This is a schematic diagram of the camera's exposure processing flow in an embodiment of this application. Detailed Implementation

[0056] The present application will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present application. Any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the protection scope of the present application.

[0057] Whether a train track has moved usually needs to be determined periodically by manual surveying to prevent train safety accidents.

[0058] If a camera is used for safety inspections of train tracks, the camera needs to clearly capture images of the tracks to accurately determine if they have shifted. However, in areas of the track far from the station, there is often no electrical circuit. Therefore, if the camera's markers are illuminated lights, and both the lights and the camera are battery-powered, the frequency and cost of equipment maintenance will increase significantly. See also Figure 1 As shown, in this embodiment of the application, a reflective target is used as a marker for calibration, thereby reducing power consumption.

[0059] See Figure 2 The reflective target shown in this embodiment is affixed to a train track. The reflective target may have at least two circular portions. These circular portions may be made of a material that strongly reflects long-wave infrared light, such as infrared reflective pigments and / or aluminum. The circular portions may also be made of a highly reflective material with waterproof properties, such as waterproof plastic or aluminum alloy. This design makes the reflective target on the train track less susceptible to rain damage, extending its service life.

[0060] The camera is aimed at the reflective target and captures an image. Exemplarily, the camera in this embodiment may include an infrared fill light, a wide dynamic range CMOS sensor, a super depth-of-field camera, and two ISPs (Image Signal Processors). The infrared fill light can be an LWIR long-wave infrared fill light with a wavelength of 8-14μm, and is placed inside the camera. Preferably, the wavelength of the infrared fill light is 950nm, at which there is no visible infrared light, which is more compatible with the infrared sensitivity of the sensor and can make circular areas in dark frame images appear more rounded. The super depth-of-field camera can be set with a fixed focal length and fixed aperture, with a depth of field range between 3 and 21 meters. The camera can be located at a distance of 3 to 21 meters from the reflective target, and the horizontal height of the camera above the ground can be between 1 and 1.5 meters.

[0061] By combining a super depth-sensing camera with infrared light, circles in dark frames can be made rounder, and irregularities can be reduced, thereby reducing the amount of subsequent calculations and processing time.

[0062] For example, the aforementioned infrared fill light can remain constantly lit during each exposure period. After the light reaches the aforementioned reflective target, it is reflected back to the camera, captured by the camera's ultra-depth-of-field camera, and then reaches the wide dynamic range CMOS sensor. The wide dynamic range CMOS sensor converts the optical signals of the two images captured under different exposure ranges into two digital signals, which are sent to two ISPs for processing respectively. The image obtained under area exposure is called the dark frame image, and the image obtained under global exposure is called the bright frame image.

[0063] Reference Figure 3 As shown, the first ISP processes the dark frame image, performs brightness statistics on the circular regions of the dark frame image, and performs automatic exposure based on the results of the brightness statistics. The first ISP can also calculate the center position of the circle using a circle center algorithm to determine whether the center has shifted.

[0064] If the center positions of the two circular areas are offset, it is determined that the train track has moved, an alarm message is generated, and a manual verification is prompted to check whether the train track has moved.

[0065] If the center positions of the two circular areas do not shift, it is determined that the train track has not moved. After storing the current exposure parameters, the camera is instructed to go into sleep mode, and the infrared lights are instructed to turn off.

[0066] The second ISP processes bright frame images, performs statistics on the brightness of the entire image, and performs automatic exposure based on the results of the brightness statistics.

[0067] For example, after initializing the camera, historical brightness and / or historical exposure values ​​are obtained for each exposure. Based on the statistical results of these historical brightness values, an appropriate brightness is selected as the exposure parameter for the current bright frame, thereby reducing the number of exposures.

[0068] In an optional embodiment of this application, after the first ISP calculates the gamma curve of the image, it can adjust the pixels according to the desired brightness Y of the circular region as follows:

[0069] 1) Brighten pixels with a brightness greater than Y in dark frames;

[0070] 2) Darken the pixels in the dark frame image whose brightness is lower than Y-(JL), where Y represents the preset desired brightness, J represents the maximum depth of field of the ultra-depth camera, and L represents the distance between the camera and the reflective target.

[0071] 3) Set the brightness value of pixels in the dark frame image whose brightness is lower than twice the black level brightness to zero.

[0072] The above three steps can be used in combination or at least one of them can be used. The order of use is not limited here.

[0073] For example, assuming the desired brightness Y is 120, the depth of field range of the ultra-depth camera is 3-21 meters, and the camera is 5 meters away from the reflective target, then pixels with a brightness lower than 120-(21-5)=104 will be darkened.

[0074] For example, the brightness value of pixels with a brightness value lower than 104 can be reduced by half, or the brightness of pixels with a brightness value lower than 104 can be reduced by one-third.

[0075] After darkening pixels with a brightness below Y-(JL), the brightness values ​​of pixels with a brightness below twice the black level can be set to zero, resulting in higher contrast. Because the black level of each wide dynamic range CMOS sensor varies, the black level brightness error between different wide dynamic range CMOS sensors may differ by -10% to 10%. Therefore, setting the brightness values ​​of pixels below twice the black level can balance improving contrast and display quality. For example, when the black level brightness of a wide dynamic range CMOS sensor is 15, the brightness values ​​of pixels with a brightness below 30 can be set to zero to further enhance contrast. This makes the circular area of ​​the target more prominent and easier to identify in the adjusted dark frame image.

[0076] For example, see Figure 4 As shown, the Y-axis represents the brightness value of an image pixel, and the X-axis represents the brightness value of an image pixel. For pixels with a brightness value higher than 120, their brightness value is adjusted to 1024 to achieve brightening. See [link to documentation]. Figure 4 The two rightmost dots. Set the brightness values ​​of pixels with a brightness level below 30 to zero, the same as the black level. See [link / reference]. Figure 4 The eight dots on the left side of the middle.

[0077] Thus, referring to Figure 4 The blue curve in the image indicates that the contrast of the corrected dark frame is higher, and the circular areas in the image become more rounded (see [link]). Figure 5 This also reduces interference from other stray light. Compared to Figure 1 In other words, Figure 5 The circular areas within become even more rounded. Figure 5 The red calibration box in the diagram can be a manually drawn calibration box used in subsequent automated calibration processes, or it can be an automatically drawn calibration box after detecting circular regions using an edge detection algorithm. In one alternative approach, after confirming that both circular regions are within the calibration box, the position of the calibration box can be fixed to facilitate subsequent detection.

[0078] See Figure 6 As shown, the gamma curve of the bright frame image is indicated by the blue line, which is used to meet the needs of a large scene range.

[0079] In this embodiment, a circular region can be displayed in an image containing a reflective target by adjusting exposure parameters. The region to be calibrated is selected, and the coordinates of this circular region are used as the reference region for the pixel coordinates of the exposed region. This is then appropriately expanded so that the pixel coordinates of the exposed region can be retained even with subsequent displacement. For example, assuming the circular region covers two circles with a diameter of 4cm, the pixel coordinates of the exposed region can be a rounded rectangle covering these two circles, with a length of 14cm and a width of 6cm. This ensures that the closest distance between the two circular regions and the edge of the calibration frame is 1cm, guaranteeing that even with slight positional changes, the exposed region still covers the circular positions within the reflective target.

[0080] The setting of two circular center regions ensures that the camera only starts detecting whether the center of a circular region is off-center when the distance between the centers of the two circular regions meets the preset center distance. Therefore, it can avoid misjudgment caused by ordinary reflective points such as garbage and glass, and avoid calculating the center position of each circular region, thus avoiding wasting computing resources.

[0081] After the reflective target is installed on the train track, the camera parameters can be set as follows:

[0082] 1) Acquire initial parameters of the camera

[0083] After the reflective target is affixed to the train track, the center position of the reflective target can be recorded as the initial calibration position of the circular area of ​​the reflective target. The expected brightness A, the actual brightness B of the circular area, and the effective exposure C (displayed as a circular area in the image captured by the camera) at the calibration time of selecting the area to be calibrated are also recorded. The initial calibration position, expected brightness A, actual brightness B, and exposure C can all be used as initial parameters for automatic exposure. In this embodiment, the expected brightness A can remain constant. The exposure represents the integral of the illuminance received by the image during the exposure time.

[0084] The camera can remain in a dormant state for extended periods, activating only based on heartbeat timing or a preset detection time.

[0085] 2) The camera powers on after the first heartbeat time or the first preset detection time has elapsed:

[0086] After the camera is powered on, before turning on the infrared lights, the ISP can first collect data to determine whether the wide dynamic range CMOS sensor is experiencing outflow. Since acquiring raw image data (RAW) requires memory to store the RAW data, and then determining whether the wide dynamic range CMOS sensor is experiencing outflow based on the RAW data, this embodiment acquires the vertical drive (VD) information of the wide dynamic range CMOS sensor and determines whether the VD information is updated. If the VD information is updated after each acquisition by the camera, it indicates that the wide dynamic range CMOS sensor is experiencing outflow.

[0087] After determining the outflow of the wide dynamic range CMOS sensor, the infrared lights are turned on. Based on the previously recorded desired brightness A and effective exposure C, the camera's initial exposure parameters are set, with an exposure adjustment interval of once per frame. This camera can acquire dark frame images in a low frame rate mode, at 12 frames per second.

[0088] In the above process, VD information can be obtained using the first ISP to determine whether the wide dynamic range CMOS sensor is emitting current, or VD information can be obtained using the second ISP to determine whether the wide dynamic range CMOS sensor is emitting current. By determining whether the VD information of the wide dynamic range CMOS sensor is updated, the decision to turn on the infrared illuminator can be made to avoid the infrared illuminator remaining on and wasting camera energy when there is no current.

[0089] When exposure stability is determined based on the VD information from the CMOS sensor, the first ISP is instructed to capture an image, acquire a dark frame image, identify circular regions in the dark frame image, and compare the center position of the circular region with the initial calibration position in the initial parameters to determine the displacement offset. The desired brightness A, the actual brightness B1 of this capture, and the effective exposure C1 of this capture are recorded. The camera then enters sleep mode again, thereby saving battery and passive camera consumption. For example, an automatic exposure algorithm can be used to determine whether the absolute value of the difference between the actual brightness and the desired brightness of three consecutive frames is less than 3. If so, the exposure is determined to be stable.

[0090] For example, after determining the outflow of the wide dynamic range CMOS sensor, the infrared illuminator can be left off. The previously recorded desired brightness (A) and effective exposure (C) can be acquired, and the camera's initial exposure parameters can be set with an exposure adjustment interval of one frame. After determining automatic exposure stabilization based on the CMOS sensor's VD information (at which point the camera can provide an exposure stabilization indicator), the infrared illuminator can be turned on. The camera waits for the next exposure to stabilize (e.g., approximately 5 frames later), then captures a dark frame image. The center position of the circle is obtained from the dark frame image, the displacement offset is calculated, and the current desired brightness (A), the actual brightness (B1) of this capture, and the effective exposure (C1) of this capture are recorded. The camera then enters sleep mode again, thus achieving more accurate capture. By only turning on the infrared illuminator when the exposure is stable, the illuminator's illumination time is shortened, saving battery power and passive camera consumption. For example, after turning on the infrared illuminator, if the absolute value of the difference between the actual brightness and the desired brightness (A) of the 3rd, 4th, and 5th frames is less than 3, then the 5th frame is considered to have stable exposure, and the 6th frame (dark frame) is captured.

[0091] 3) The camera will power on after the second heartbeat time or the second preset detection time has elapsed.

[0092] After the camera is powered on, before turning on the infrared lights, it's necessary to first check if the wide dynamic range CMOS sensor is experiencing outflow. The vertical drive (VD) information of the wide dynamic range CMOS sensor is acquired, and it's determined whether the VD information has been updated. If the VD information has been updated, it indicates that the wide dynamic range CMOS sensor is experiencing outflow. At this point, the infrared lights are turned on, and the camera's initial exposure parameters are set based on the previously recorded desired brightness A and effective exposure C1. Once the exposure is determined to be stable based on the CMOS sensor's VD information, the first ISP is instructed to capture an image, acquiring a dark frame image. Circular regions in the dark frame image are identified, and the center position of these regions is compared with the initial calibration position in the initial parameters to determine the displacement offset. The desired brightness A, the actual brightness B2 of this capture, and the effective exposure C2 of this capture are recorded. The camera then enters sleep mode again.

[0093] For example, after determining the outflow of the wide dynamic range CMOS sensor, the infrared light can be turned off initially. The previously recorded desired brightness A and effective exposure C can be obtained, and the camera's initial exposure parameters can be set with an exposure adjustment interval of once per frame. After the automatic exposure stabilizes, the infrared light can be turned on, and the camera can wait for the next exposure to stabilize (for example, approximately 5 frames later). Then, a dark frame image can be captured. The center position of the circle can be obtained from the dark frame image, the displacement offset can be calculated, and the current desired brightness A, the actual brightness B1 of this capture, and the effective exposure C1 of this capture can be recorded. The camera then enters sleep mode again, thereby achieving more accurate capture. By turning on the infrared light only when the exposure is stable, the illumination time of the infrared light can be shortened, saving battery and passive camera consumption.

[0094] In this process, the pixel difference between the detected center positions at different times can be determined as the displacement offset. The actual physical size of the circular area in the reflective target is compared with the displacement offset to establish the relationship between pixels and distance, thereby calculating the actual offset distance of the reflective target.

[0095] For example, assuming a 4-second interval, the two dark frame images captured are the first and second dark frame images. There is a 3-pixel difference between the center of the circle detected in the first dark frame image and the center of the circle detected in the second dark frame image. Therefore, the actual distance between the two centers can be calculated based on the actual size of the reflective target. For example, if the distance between the two centers of the reflective target is 8cm, and the two centers are 220 pixels apart in the dark frame image, then 1 pixel corresponds to an actual distance of 8cm / 220 = 0.0364cm, or 0.364mm. Therefore, the actual offset distance of the center position during the two measurements is 0.364mm * 3 = 1.092mm.

[0096] For example, the detection of circular regions in dark frame images can be achieved, but is not limited to, using at least one of the following: Hough Transform, Contour-Based Detection, RANSAC circle fitting algorithm, or deep learning.

[0097] Optionally, Hough transform is used to detect circular regions. After determining that a target is a reflective object by the distance between the centers of two circular regions, the center position is found by Hough transform. Then, the position of the center is determined by the offset distance of the center position at different times. Since Hough transform is an algorithm based on an accumulator voting mechanism, even if the edge of the circular region is missing due to factors such as occlusion or uneven lighting, as long as there are enough edge points that conform to the geometric features of a circle, the circular region can be detected. Therefore, using Hough transform to detect circular regions can enable the camera to find reflective objects more accurately, thereby improving the accuracy of the camera in detecting whether the train track is moving.

[0098] After detecting a circular region, the location of the center can be determined by, but is not limited to, at least one of the following methods: perpendicular bisector, folding, right angle, etc.

[0099] The preset detection time can include daytime and nighttime. During nighttime, the camera captures dark frames to detect the position of the reflective target. When it is determined that the position of the reflective target has shifted, the displacement amount is recorded.

[0100] At night, when ambient brightness is relatively constant, image quality mainly relies on infrared lights. Since the light source is relatively singular, the image quality is also more consistent.

[0101] Because dark frames at night are more affected by infrared light and less affected by natural light, the circles captured in dark frame images are rounder, and the circular areas detected by the above method are also rounder. Furthermore, in this embodiment, the gamma curve of the dark frame image is corrected by increasing contrast. When the corrected dark frame image data represents the train track without movement, the current exposure parameters are stored. These current exposure parameters include the current brightness expectation, actual brightness, and effective exposure. The resulting exposure parameters produce a clearer image and have higher reference value for subsequent exposures. The camera exposure parameters at this time are also more beneficial for subsequent camera calibration.

[0102] The time before a train passes by during the day can also be used as a preset detection time. After the camera is turned on, it is determined whether the position of the reflective target has shifted based on the dark frame. If so, double exposure is initiated.

[0103] In double exposure mode, the camera can perform area exposure and wide-range exposure on a reflective target to obtain dark frame images and bright frame images respectively.

[0104] In an optional embodiment of this application, when there is natural light during the day, the brightness, shutter speed, gain, and illuminance parameters of the automatic exposure of the bright frame can be used to determine whether the current environment is in a strong light state. In a strong light state, the infrared lamp does not need to be turned on. Therefore, when the ambient light intensity reaches a strong light state, the camera can not turn on the infrared lamp for supplementary lighting, thus saving power consumption.

[0105] The following logic can be used to determine whether the current environment is under strong light:

[0106] The current shutter speed is represented by SW, the current gain by GAIN, the current brightness by Y, and the current illuminance by LV.

[0107] If the current environment simultaneously meets the following four conditions, it is determined that the current environment is in a state of strong light.

[0108] 1) SW < 2 * sw_min

[0109] 2) GAIN = gain_min;

[0110] 3) The difference between Y and the expected brightness of the bright frame is within ±10;

[0111] 4) The current estimated illuminance (LV) is greater than 60.

[0112] Where sw_min is the minimum shutter speed (exposure time) supported by the current camera, such as 10ms, 8ms, etc., and gain_min represents the minimum gain supported by the current camera. The expected brightness of the bright frame is the expected brightness carried in the camera's automatic exposure parameters. For example, it can be the expected brightness in the automatic exposure parameters of the automatic exposure algorithm stored in the second ISP.

[0113] The current estimated illuminance can be obtained from the current gain, current shutter speed, and current brightness. For example, the current estimated illuminance (LV) can be obtained using the following three formulas:

[0114] lv1=(real_gain*sw) / log10(4*exp(Y))(1)

[0115] Where lv1 represents the first intermediate variable for estimating illuminance, real_gain represents the current gain, sw represents the current shutter speed, and Y represents the current brightness.

[0116] lv2 = log(lv1 * 1000 + 1);

[0117]

[0118] Where LV represents the current estimated illuminance, and lv2 represents the second intermediate variable for estimating illuminance.

[0119] For example, in this embodiment of the application, when real_gain and sw take the following values, the result calculated by LV is as follows.

[0120] real_gain=1,sw=1,Y=255LV=99

[0121] real_gain=1,sw=100,Y=100LV=70

[0122] real_gain=1,sw=200,Y=100LV=66

[0123] real_gain=50,sw=5000,Y=100LV=26

[0124] real_gain=100,sw=10000,Y=3LV=1

[0125] Because there is natural light during the day, the energy attenuation of the supplementary light is not obvious. However, at night, imaging relies mainly on supplementary light, so the energy attenuation of the supplementary light is more significant.

[0126] For example, in the embodiments of this application, when it is determined that the actual brightness is gradually decreasing, the exposure can be increased, and by recording the parameters and time of night capture, it can be determined whether the effective exposure is slowly increasing. If the effective exposure is always slowly increasing, it is determined that the battery is degrading. If the effective exposure suddenly increases and then remains stable, it is determined that the camera is dirty. If there is a defect in the circular area of ​​the dark frame image, it is determined that the reflective target is dirty.

[0127] Once it is determined that the battery is deteriorating or the reflector or camera is dirty, the user can be notified to replace the battery or manually remove the dirt. Because there is no natural light interference at night, the contrast of dark frames is higher. Therefore, more dark frame images can be collected at night to calculate the trend of exposure changes, which can be used to determine whether the battery is deteriorating or whether there is dirt in the camera or reflector.

[0128] As shown in the table below, C1-C49 indicates a gradual increase in effective exposure, which indicates that the battery is slowly degrading. C49-C50 indicates a sudden increase of 5 times in effective exposure. C49-C51 indicates a sudden and significant increase in effective exposure followed by a gradual leveling off, which indicates that the camera has a large area of ​​dirt.

[0129]

[0130] Table 1

[0131] Based on the historical trend of effective exposure at night, if it is determined that the camera or reflective target is dirty or the battery is degraded, the camera's location information and the judgment result can be sent to the user as a prompt. Bright frame images taken during the day can also be sent to the user as a prompt, so that the camera can continue to be used for a period of time after the battery degrades. This avoids the problem of being unable to calibrate or having unclear images after the battery degrades. It also makes it convenient for users to carry maintenance batteries or cleaning tools according to the battery condition or the degree of dirt, avoiding repeated trips to the train tracks, saving manpower, improving efficiency, and reducing the risk of accidents during equipment installation.

[0132] Reference Figure 7 As shown, this application provides a method for detecting track displacement, including:

[0133] Step 701: After the camera is started, detect whether the vertical drive VD information of the wide dynamic range CMOS sensor is updated through the first ISP and / or the second ISP. After the VD information is updated, turn on the infrared light.

[0134] Step 702: Acquire optical signals from the reflective target using an ultra-depth-of-field camera, and transmit the optical signals to a wide dynamic range CMOS sensor.

[0135] The reflective targets are located on the train tracks.

[0136] Step 703: Acquire dark frame images under regional exposure and bright frame images under global exposure using a wide dynamic range CMOS sensor, transmit the dark frame images to the first ISP, and transmit the bright frame images to the second ISP.

[0137] Step 704: Correct the gamma curve of the dark frame image by increasing contrast using the first ISP: when the corrected dark frame image data represents the train track not moving, store the current exposure parameters, which include the current brightness expectation, actual brightness, and effective exposure; when the corrected dark frame image data represents the train track moving, generate alarm information.

[0138] Step 705: Acquire bright frame images as monitoring data through the second ISP.

[0139] Figure 8 This is a schematic diagram of the camera's exposure processing flow in an embodiment of this application. After startup, the camera uses a wide dynamic range CMOS sensor and two different exposure modes: exposure A for a specific area and exposure B for the entire system, resulting in dark frame A and bright frame B. Dark frame A undergoes center detection via a first ISP, while bright frame B is monitored for scene observation via a second ISP.

[0140] This application also provides a track displacement detection system, including a camera and a reflective target located on a train track.

[0141] The camera includes: an infrared LED, a fixed-focus, fixed-aperture ultra-depth-of-field camera, a wide dynamic range CMOS sensor, a first ISP, and a second ISP;

[0142] After the camera is started, it checks whether the vertical drive VD information of the wide dynamic range CMOS sensor has been updated. After the VD information is updated, the infrared light is turned on.

[0143] The ultra-depth-of-field camera is used to aim at a reflective target and collect optical signals, which are then transmitted to a wide dynamic range CMOS sensor.

[0144] The wide dynamic range CMOS sensor is used to acquire dark frame images under regional exposure and bright frame images under global exposure, and transmits the dark frame images to the first ISP and the bright frame images to the second ISP.

[0145] The first ISP is used to correct the gamma curve of the dark frame image by increasing the contrast. When the corrected dark frame image data represents the train track without movement, it stores the current exposure parameters, which include the current brightness expectation, actual brightness and effective exposure.

[0146] The second ISP is used to acquire bright frame images as monitoring data.

[0147] Optionally, the second ISP can also store current exposure parameters, including the current brightness expectation, actual brightness, and effective exposure, for later reference. Alternatively, the second ISP can also obtain historical exposure parameters stored by the first ISP, such as historical brightness expectations, actual brightness, and effective exposure.

[0148] This monitoring data is for train track scenarios.

[0149] Optionally, the first ISP, in the process of correcting the gamma curve of the dark frame image in a way that increases contrast, is also used for:

[0150] Brighten pixels in dark frames that have a brightness greater than the preset desired brightness.

[0151] Darken pixels in dark frames whose brightness is less than twice the black level brightness.

[0152] Optionally, the reflective target includes two circular areas made of a reflective material, wherein the reflective material includes infrared reflective pigments and / or aluminum.

[0153] The first ISP is also used to detect two circular regions in the dark frame image by at least one of Hough Transform, Contour-Based Detection, RANSAC circle fitting algorithm or deep learning after correcting the gamma curve of the dark frame image in a way that increases contrast.

[0154] The center positions of two circular regions can be detected by at least one of the following methods: perpendicular bisector method, folding method, and right angle method.

[0155] Optionally, the first ISP is also used for:

[0156] Obtain the distance between the centers of two circular regions;

[0157] If the center distance between two circular regions matches the preset center distance, determine whether the center positions of the two circular regions are offset:

[0158] If the center positions of the two circular areas are offset, it is determined that the train track has moved, an alarm message is generated, prompting manual verification of whether the train track has moved, and the second ISP is notified to store the bright frame image within a preset time period.

[0159] If the center positions of the two circular areas do not shift, it is determined that the train track has not moved. After storing the current exposure parameters, the camera is instructed to go into sleep mode, and the infrared lights are instructed to turn off.

[0160] After determining that the center position has shifted, the second ISP is instructed to store the bright frame data. This way, before dispatching personnel for on-site verification, the bright frame data stored in the second ISP can be used to verify the track offset before dispatching personnel, which greatly improves the accuracy of safety detection.

[0161] Optionally, the camera is also used for:

[0162] The camera is activated at a preset detection time, including nighttime hours, by heartbeat monitoring. It detects whether the vertical drive VD information of the wide dynamic range CMOS sensor has been updated. After the VD information is updated, the infrared light is turned on, and the exposure is performed based on the expected brightness, actual brightness, and effective exposure based on historical records.

[0163] Optionally, the camera also includes a battery, and the first ISP is also used for...

[0164] Increase the effective exposure when the actual brightness decreases steadily in historical records;

[0165] Determine if the battery is degrading based on the historical trend of effective exposure at nighttime.

[0166] After confirming that the effective exposure is slowly increasing, the system determines that the battery is depleting and sends the camera's location information and battery status as a notification to the user.

[0167] For example, the second ISP can also store the current exposure parameters, which include the current brightness expectation, actual brightness, and effective exposure.

[0168] Increase the effective exposure when the actual brightness decreases steadily in historical records;

[0169] Determine if the battery is degrading based on the historical trend of effective exposure at nighttime.

[0170] After confirming that the effective exposure is slowly increasing, the system determines that the battery is depleting and sends the camera's location information and battery status as a notification to the user.

[0171] Optionally, the camera also includes a battery, and the first or second ISP is also used for,

[0172] Increase the effective exposure when the actual brightness decreases steadily in historical records;

[0173] Based on the historical trend of effective exposure at nighttime, determine whether the camera is dirty;

[0174] After confirming that the exposure has stabilized after a sharp increase, and determining that the camera is dirty, the camera's location information and a bright frame image taken during the day are sent to the user as a notification.

[0175] Optionally, the depth-of-field range of the ultra-depth camera is 3 to 21 meters, the distance between the reflective target and the camera is 3 to 21 meters, and the infrared light is an LWIR long-wave infrared fill light with a wavelength of 8-14μm. Preferably, the wavelength of the infrared fill light is 950nm, at which there is no visible infrared light, which is more compatible with the infrared light sensitivity of the sensor and can make circular areas in dark frame images more rounded.

[0176] Optionally, the camera's frame rate is 12 frames per second.

[0177] In this embodiment, exposure is determined by checking whether the VD information of the wide dynamic range CMOS sensor is updated, avoiding wasting camera energy when exposure fails. Double exposure is performed using the wide dynamic range CMOS sensor, and the current exposure parameters are stored when the dark frame image data indicates that the train track has not moved, reducing memory consumption. Furthermore, optical signals are acquired using a reflective target, eliminating the need for additional light sources and frequent manual calibration, thus efficiently improving the shooting effect of cameras near the train track and reducing overall energy consumption. Since determining whether the train track has moved is more accurate using the calibrated dark frame image, repeated manual surveying is unnecessary, greatly improving the efficiency of train track detection. This achieves efficient, low-cost, and automated train track safety detection, preventing accidents caused by train track movement and ensuring safe train travel.

[0178] The aforementioned memory can be a storage device, such as random access memory, read-only memory, non-volatile memory, programmable ROM, erasable PROM, electrically erasable memory, flash memory, optical memory, and registers. The processor can be a general-purpose processor, which executes specific steps and / or operations by reading and executing computer programs stored in the memory. The general-purpose processor may utilize the stored memory during the execution of these steps and / or operations. The general-purpose processor can be a central processing unit, ASIC, or FPGA, etc. In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor.

[0179] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0180] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0181] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above descriptions are merely preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A track displacement detection system, characterized in that, Includes a camera and a reflective target located on a train track. The camera includes: an infrared lamp, a fixed-focus, fixed-aperture ultra-depth-of-field camera, a wide dynamic range CMOS sensor, a first ISP, and a second ISP. After the camera is started, it detects whether the vertical drive VD information of the wide dynamic range CMOS sensor is updated through the first ISP and / or the second ISP. After the VD information is updated, the infrared light is turned on. The ultra-depth-of-field camera is used to aim at the reflective target and collect optical signals to transmit to the wide dynamic range CMOS sensor. The wide dynamic range CMOS sensor is used to acquire dark frame images under regional exposure and bright frame images under global exposure, and transmit the dark frame images to the first ISP and the bright frame images to the second ISP. The first ISP is used to correct the gamma curve of the dark frame image by increasing contrast: when the corrected dark frame image data indicates that the train track has not moved, it stores the current exposure parameters, which include the current brightness expectation, actual brightness, and effective exposure; when the corrected dark frame image data indicates that the train track has moved, it generates alarm information. The second ISP is used to acquire bright frame images as monitoring data.

2. The system as described in claim 1, characterized in that, In the process of correcting the gamma curve of the dark frame image by increasing contrast, the first ISP is also used to: Brighten pixels in the dark frame image that have a brightness greater than Y; and / or In the dark frame image, pixels with brightness lower than Y-(JL) are darkened, where Y represents the preset desired brightness, J represents the maximum depth of field of the ultra-depth-of-field camera, and L represents the distance between the camera and the reflective target; and / or Set the brightness value of pixels in the dark frame image whose brightness is lower than twice the black level brightness to zero.

3. The system as described in claim 1, characterized in that, The reflective target includes two circular areas made of reflective material, wherein the reflective material includes infrared reflective pigment and / or aluminum; The first ISP is further configured to detect two circular regions in the dark frame image by means of at least one of Hough Transform, Contour-Based Detection, RANSAC circle fitting algorithm or deep learning after correcting the gamma curve of the dark frame image in a manner that increases contrast. The center positions of two circular regions can be detected by at least one of the following methods: perpendicular bisector method, folding method, and right angle method.

4. The system as described in claim 1, characterized in that, The first ISP is also used for: Obtain the distance between the centers of two circular regions; When the center distance between the two circular regions meets the preset center distance, determine whether the center positions of the two circular regions are offset: If the center positions of the two circular areas are offset, it is determined that the train track has moved, an alarm message is generated, prompting manual verification of whether the train track has moved, and the second ISP is notified to store the bright frame image within a preset time period. If the center positions of the two circular areas do not shift, it is determined that the train track has not moved. After storing the current exposure parameters, the camera is instructed to go into sleep mode, and the infrared lights are instructed to turn off.

5. The system as described in claim 1, characterized in that, The first ISP or the second ISP is also used for: The camera is activated at a preset detection time, including nighttime hours, by heartbeat monitoring. The vertical drive VD information of the wide dynamic range CMOS sensor is updated by the first ISP and / or the second ISP. After the VD information is updated, the infrared light is turned on, and the wide dynamic range CMOS sensor is controlled to perform exposure based on the historical expected brightness, actual brightness, and effective exposure.

6. The system as described in claim 5, characterized in that, The camera also includes a battery, and the first or second ISP is further used for: When the actual brightness steadily decreases in the historical records, the effective exposure amount is increased; Based on the trend of changes in effective exposure at nighttime in the historical records, determine whether the battery is degrading; After determining that the effective exposure is slowly increasing, the battery is judged to be degrading, and the camera's location information and battery status are sent to the user as a prompt message.

7. The system as described in claim 5, characterized in that, The camera also includes a battery, and the first or second ISP is further used for: When the actual brightness steadily decreases in the historical records, the effective exposure amount is increased; Based on the trend of changes in effective exposure at nighttime in the historical records, determine whether the camera is dirty; After confirming that the effective exposure has stabilized after a sharp increase, and determining that the camera is dirty, the camera's location information and a bright frame image taken during the day are sent to the user as a notification.

8. The system as described in any one of claims 1-7, characterized in that, The depth-of-field range of the ultra-depth camera is 3 to 21 meters, the distance between the reflective target and the camera is 3 to 21 meters, the frame rate of the camera is 12 frames per second, and the infrared light is an LWIR long-wave infrared fill light with a wavelength of 950nm.

9. A method for detecting track displacement, characterized in that, include: After the camera is activated, the vertical drive (VD) information of the wide dynamic range CMOS sensor is checked via the first ISP and / or the second ISP. If the VD information is updated, the infrared LED is turned on. Optical signals are acquired from a reflective target using an ultra-depth-of-field camera and transmitted to the wide dynamic range CMOS sensor, wherein the reflective target is located on a train track. The wide dynamic range CMOS sensor acquires dark frame images under regional exposure and bright frame images under global exposure, and transmits the dark frame images to the first ISP and the bright frame images to the second ISP. The first ISP corrects the gamma curve of the dark frame image by increasing contrast. When the corrected dark frame image data represents the train track without movement, the current exposure parameters are stored. The current exposure parameters include the current brightness expectation, actual brightness, and effective exposure. Bright frame images are acquired as monitoring data through the second ISP.

10. A camera, characterized in that, include: Fixed-focus, fixed-aperture ultra-depth-of-field camera, wide dynamic range CMOS sensor, first ISP and second ISP; After the camera is powered on, it detects whether the vertical drive VD information of the wide dynamic range CMOS sensor has been updated via the first ISP and / or the second ISP. After the VD information is updated, the infrared light is turned on. The ultra-depth-of-field camera is used to aim at a reflective target and collect optical signals to transmit to the wide dynamic range CMOS sensor, wherein the reflective target is located on the train track; The wide dynamic range CMOS sensor is used to acquire dark frame images under regional exposure and bright frame images under global exposure, and transmits the dark frame images to the first ISP and the bright frame images to the second ISP. The first ISP is used to correct the gamma curve of the dark frame image in a way that increases contrast: when the corrected dark frame image data indicates that the train track has not moved, it stores the current exposure parameters, which include the current brightness expectation, actual brightness, and effective exposure; when the corrected dark frame image data indicates that the train track has moved, it generates an alarm message. The second ISP is used to acquire bright frame images as monitoring data.

Images