Video signal-based vehicle motion state detection method and system

Abstract

The invention provides a vehicle motion state detection method and system based on a video signal. The method comprises the following steps: acquiring a driving view video image sequence in front of a target train in a driving process; detecting the motion state of the target train based on pixel information between frames of images in the driving view video image sequence and a target motion state detection mode to obtain a vehicle motion state detection result of the target train; wherein the target motion state detection mode is determined from a plurality of motion state detection modes according to the light environment type of the driving view video image sequence; the plurality of motion state detection modes are constructed according to the brightness degree corresponding to the light environment type. According to the method, the pixel information of the continuous frame images in front of the running train in the luminous environment and the dark environment is analyzed, so that the motion state of the train is accurately obtained, and the detection speed is increased.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit, and in particular, to a method and system for detecting the motion state of a vehicle based on video signals. Background Art

[0002] Since a subway cannot receive satellite signals when running underground and cannot use GPS for positioning, and there may also be signal interruptions in rainy or cloudy days with thick clouds. In order to detect the current position of the subway, inertial navigation technology is usually used for detection. However, this method has certain errors and needs to be calibrated each time the subway arrives at the platform. It is necessary to detect the motion state of the subway to assist in calibration. When the detected motion state of the subway is stationary, it can be indicated that the subway has arrived at the platform, thus assisting the inertial navigation technology for calibration.

[0003] Currently, for the method of detecting the motion state of a subway, it mainly uses shear sensors, infrared pair sensors, passive and active sensor groups to measure signals, and then a computing device calculates the running state of the train. It is necessary to install additional sensor devices on the subway. At the same time, due to the large amount of calculation, there are also problems of low detection efficiency and low detection accuracy.

[0004] Therefore, there is an urgent need for a method and system for detecting the motion state of a vehicle based on video signals to solve the above problems. Summary of the Invention

[0005] In view of the problems existing in the prior art, the present invention provides a method and system for detecting the motion state of a vehicle based on video signals.

[0006] The present invention provides a method for detecting the motion state of a vehicle based on video signals, including: Obtaining a sequence of video images of the driving vision in front of a target train during driving; Detecting the motion state of the target train based on the pixel information between each frame of the sequence of video images of the driving vision and a target motion state detection mode, to obtain a detection result of the motion state of the vehicle of the target train; Wherein, the target motion state detection mode is determined from a plurality of motion state detection modes according to the light environment type of the sequence of video images of the driving vision; the plurality of motion state detection modes are constructed according to the brightness degree corresponding to the light environment type.

[0007] According to the method for detecting the motion state of a vehicle based on video signals provided by the present invention, the light environment type is determined through the following steps: Calculating the gray value corresponding to each frame of image according to the pixel value of each frame of the sequence of video images of the driving vision; Determine the light environment type corresponding to the driving vision video image sequence according to the gray value and the preset brightness threshold; Among them, the light environment types include bright environment, dim environment, and lightless environment; the multiple motion state detection modes include a first motion state detection mode and a second motion state detection mode. The first motion state detection mode is used to detect the driving vision video image sequence in the bright environment or the dim environment; the second motion state detection mode is used to detect the driving vision video image sequence in the lightless environment.

[0008] According to a vehicle motion state detection method based on video signals provided by the present invention, the preset brightness threshold includes a first brightness threshold and a second brightness threshold, where the first brightness threshold is greater than the second brightness threshold; The step of determining the light environment type corresponding to the driving vision video image sequence according to the gray value and the preset brightness threshold includes: If the gray value is greater than the first brightness threshold, determine that the light environment type of the driving vision video image sequence is the bright environment; If the gray value is less than or equal to the first brightness threshold and greater than the second brightness threshold, determine that the light environment type of the driving vision video image sequence is the dim environment; If the gray value is less than or equal to the second brightness threshold, determine that the light environment type of the driving vision video image sequence is the lightless environment.

[0009] According to a vehicle motion state detection method based on video signals provided by the present invention, detecting the motion state of the target train based on the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode to obtain the vehicle motion state detection result of the target train includes: When it is determined that the light environment type is the bright environment or the dim environment, based on the first motion state detection mode, judge the pixel displacement distance between adjacent frame images in the driving vision video image sequence; If the pixel displacement distance of the minimum displacement pixel point between the adjacent frame images is less than or equal to the minimum displacement threshold, and the pixel displacement distance of the maximum displacement pixel point between the adjacent frame images is less than or equal to the maximum displacement threshold, determine that the vehicle motion state detection result is the stationary state; If the pixel displacement distance of the maximum displacement pixel point between the adjacent frame images is greater than the maximum displacement threshold, determine that the vehicle motion state detection result is the motion state.

[0010] A method for detecting the motion state of a vehicle based on a video signal according to the present invention, which detects the motion state of the target train based on the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode, and obtains the vehicle motion state detection result of the target train, further includes: When determining that the light environment type is the bright environment or the dim environment, based on the first motion state detection mode, judge the pixel displacement distance between consecutive multiple frames of the driving vision video image sequence; If the pixel displacement distance of all the minimum displacement pixel points between the consecutive multiple frames is less than or equal to the minimum displacement threshold, and the pixel displacement distance of all the maximum displacement pixel points between the consecutive multiple frames is less than or equal to the maximum displacement threshold, determine that the vehicle motion state detection result is the stationary state; If the pixel displacement distance of all the maximum displacement pixel points between the consecutive multiple frames is greater than the maximum displacement threshold, determine that the vehicle motion state detection result is the motion state.

[0011] A method for detecting the motion state of a vehicle based on a video signal according to the present invention, which detects the motion state of the target train based on the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode, and obtains the vehicle motion state detection result of the target train, further includes: When determining that the light environment type is the lightless environment, based on the second motion state detection mode, judge the binary pixel values between the first target frame image and the second target frame image in the driving vision video image sequence, where the first target frame image is the current frame image in the driving vision video image sequence, and the second target frame image is an image in the driving vision video image sequence that is at least 5 frames apart from the first target frame image; If the binary image coincidence ratio between the first target frame image and the second target frame image is greater than or equal to the preset coincidence ratio threshold, determine that the vehicle motion state detection result is the stationary state; If the binary image coincidence ratio between the first target frame image and the second target frame image is less than the preset coincidence ratio threshold, determine that the vehicle motion state detection result is the motion state; Wherein, the binary image coincidence ratio is calculated based on the binary pixel values between the first target frame image and the second target frame image.

[0012] A method for detecting the motion state of a vehicle based on a video signal according to the present invention, the binary image coincidence ratio is calculated through the following steps: Perform binarization processing on the first target frame image to obtain a first binarized image; perform binarization processing on the second target frame image to obtain a second binarized image; Obtain the sum result of the first pixel points and the sum result of the second pixel values, where the sum result of the first pixel points is the sum of the binarized pixel values of each pixel point in the first binarized image, and the sum result of the second pixel points is the sum of the binarized pixel values of each pixel point in the second binarized image; Based on the pixel coordinates in the first binarized image and the second binarized image, perform an AND operation on the pixel values corresponding to the same pixel coordinates in the first binarized image and the second binarized image in sequence to construct a third binarized image; Obtain the sum result of the third pixel points, where the sum result of the third pixel points is the sum of the binarized pixel values of each pixel point in the third binarized image; Obtain a first coincidence ratio according to the ratio between the sum result of the third pixel points and the sum result of the first pixel points; obtain a second coincidence ratio according to the ratio between the sum result of the third pixel points and the sum result of the second pixel points; Obtain the coincidence ratio of the binarized images according to the ratio between the first coincidence ratio and the second coincidence ratio.

[0013] According to a vehicle motion state detection method based on video signals provided by the present invention, the method further includes: Based on the correspondence between the light environment type and the preset image cropping area, crop each frame of the driving vision video image sequence to obtain a cropped driving vision video image sequence, where the ranges of the preset image cropping areas corresponding to the bright environment, the dim environment, and the lightless environment increase in sequence; The detecting the motion state of the target train based on the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode to obtain the vehicle motion state detection result of the target train includes: Based on the pixel information and the target motion state detection mode between each frame of the cropped driving vision video image sequence, detect the motion state of the target train to obtain the vehicle motion state detection result of the target train.

[0014] The present invention also provides a vehicle motion state detection system based on video signals, including: A video image acquisition module for acquiring a driving vision video image sequence in front of the target train during driving; A train motion state detection module is used to detect the motion state of the target train based on the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode, and obtain the vehicle motion state detection result of the target train; Wherein, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the driving vision video image sequence; the multiple motion state detection modes are constructed according to the brightness degree corresponding to the light environment type.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the vehicle motion state detection method based on video signals as described in any one of the above.

[0016] The vehicle motion state detection method and system based on video signals provided by the present invention analyze the pixel information of consecutive frame images in front of the train during the light and dark environments, thereby accurately obtaining the motion state of the train and improving the detection speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0018] Figure 1 It is a flowchart of the vehicle motion state detection method based on video signals provided by the present invention; Figure 2 It is a structural diagram of the vehicle motion state detection system based on video signals provided by the present invention; Figure 3 It is a structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0020] As the subway has become a means of transportation chosen by more and more people when traveling, more detection algorithms need to be configured on the subway to ensure its normal operation. When the subway runs underground, it cannot receive satellite signals, so GPS cannot be used for precise positioning. In addition, in rainy days or when the clouds are thick, the signal may be interrupted, resulting in an inability to accurately know the specific location of the subway after it leaves the platform.

[0021] To solve the problem of subway positioning, the use of inertial navigation technology has been proposed currently. However, there is a certain error accumulation in this technology, and it needs to be calibrated regularly to ensure accuracy. To assist in the calibration of inertial navigation technology, related technologies need to detect whether the subway is in a stationary state. When it is determined that the subway is in a stationary state, it means that the subway has reached the platform. At this time, calibration can be carried out to correct the errors of the inertial navigation system and more accurately grasp the real-time position of the subway.

[0022] Currently, the detection technologies for the subway's motion state cover a variety of methods, mainly including several categories such as computer vision, sound recognition, machine learning, and sensor detection.

[0023] In the field of computer vision, high-precision sensors such as cameras and lidars equipped on the subway can be used to capture the surrounding environment information during the train's movement. Subsequently, with the help of image processing technology and target tracking algorithms, the motion state of the subway can be analyzed and recognized.

[0024] The sound recognition method is to capture the sound signals during the subway's movement through sound collection devices such as microphones installed on the train. By processing and analyzing these sound signals, the motion state of the subway can also be judged.

[0025] Machine learning technology uses deep learning algorithms to deeply mine and analyze the data collected by various sensors on the subway, so as to achieve intelligent recognition of the subway's motion state.

[0026] In addition, the sensor detection method is also a commonly used subway motion state detection technology. This technology relies on various sensors installed on the subway, such as gyroscopes, and directly measures the physical motion parameters of the subway to accurately judge the motion state of the subway.

[0027] However, the existing subway motion state detection methods generally face challenges such as high development costs, large equipment investment, or high training costs. To address the problems of the existing technology, the present invention provides a vehicle motion state detection method based on video signals, which can directly utilize the camera resources commonly equipped on subways without adding additional sensors, and can detect the motion state of the subway only through the images captured by the camera. At the same time, an optical flow detection algorithm is introduced and combined with binary image processing technology in low-light environments to ensure rapid and accurate vehicle state detection under different light conditions. Moreover, in the face of complex and changeable operating environments, the present invention can exhibit good robustness, providing reliable technical support for the real-time monitoring of the subway motion state.

[0028] Figure 1 FIG. is a schematic flow chart of the vehicle motion state detection method based on video signals provided by the present invention, as Figure 1 shown, the present invention provides a vehicle motion state detection method based on video signals, including: Step 101, obtaining a sequence of video images of the driving vision in front of the target train during driving.

[0029] During the operation of the train, in order to ensure safety and monitor the driving state of the train, cameras are usually installed at the front end of the train to capture real-time video images in front of the train. These cameras continuously record the vision on the train's driving path, generating a series of video images, which are arranged in chronological order to form a sequence of video images. The present invention takes the subway driving process as an example for illustration, and determines whether the subway is in a normal driving state by obtaining the situation of the road in front of the subway.

[0030] Step 102, detecting the motion state of the target train based on the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode, to obtain the vehicle motion state detection result of the target train; wherein, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the driving vision video image sequence; the multiple motion state detection modes are constructed according to the brightness level corresponding to the light environment type.

[0031] In the present invention, the sequence of video images during the subway driving process is composed of a series of consecutive frames, and each frame captures a large amount of information contained in a certain moment of the subway driving vision, and this information can be used to judge the motion state of the subway.

[0032] Further, in order to detect the motion state of the subway, the present invention analyzes the pixel information between these frames, and the change in pixel information can reflect the motion of objects in the image. At the same time, since different light environments will have a significant impact on the image quality and reduce the detection accuracy of the motion state, the present invention also needs to determine a suitable motion state detection mode according to the type of light environment.

[0033] In the present invention, multiple motion state detection modes are pre-constructed. These motion state detection modes are designed for different types of light environments, and each motion state detection mode takes into account the characteristics of the image under the corresponding light environment to ensure that the motion state of the train can be accurately detected under specific light conditions. During the actual detection process, the present invention selects the most suitable motion state detection mode from multiple pre-constructed modes according to the type of light environment of the current video image sequence. Then, the selected motion state detection mode is used to analyze the pixel information between consecutive frames, thereby obtaining the detection result of the motion state of the subway.

[0034] The vehicle motion state detection method based on video signals provided by the present invention analyzes the pixel information of consecutive frame images in front of the train during driving in a lighted environment and a dark environment, thereby accurately obtaining the motion state of the train and improving the detection speed.

[0035] Based on the above embodiments, the type of light environment is determined through the following steps: According to the pixel values of each frame image in the driving vision video image sequence, the gray value corresponding to each frame image is calculated; According to the gray value and a preset brightness threshold, the type of light environment corresponding to the driving vision video image sequence is determined; Wherein, the type of light environment includes a bright environment, a dim environment, and a dark environment; the multiple motion state detection modes include a first motion state detection mode and a second motion state detection mode. The first motion state detection mode is used to detect the driving vision video image sequence in the bright environment or the dim environment; the second motion state detection mode is used to detect the driving vision video image sequence in the dark environment.

[0036] In the present invention, each frame image in the video image sequence is composed of pixel points, and each pixel point has a color value, which is usually represented by the values of three channels: red, green, and blue (RGB). To simplify subsequent processing, the present invention converts these color images into gray images, that is, each pixel point has only one gray value (usually in the range of 0 to 255), and the higher the gray value, the brighter it is.

[0037] In order to determine the type of light environment, the present invention also pre-constructs corresponding brightness thresholds, which can be obtained through experiments or experiences related to the track-running cities and lines, and are used to distinguish bright environments, dim environments, and lightless environments. For example, if the average gray value of a frame of image is higher than a certain higher threshold (such as 125), it is determined that the current environment is bright; if the average gray value is between two thresholds (such as 60 to 125), it is determined that the current environment is a dim environment; if the average gray value is lower than the lower threshold (such as 60), it is determined that the current environment is a lightless environment, which usually means that the image is very dark and almost no details can be seen. It should be noted that in the present invention, the type of light environment of multiple consecutive frames of images (such as consecutive 5 frames or consecutive 10 frames) can be judged. When it is currently determined that all consecutive frame images belong to a certain type of light environment, it can be determined that the current driving vision video image sequence also belongs to this type of light environment.

[0038] In the present invention, the first motion state detection mode is applicable to bright or dim environments. In this mode, the present invention can use the image to perform optical flow calculation to obtain the optical flow data in the image and the displacement distance information of each pixel point, so as to detect the movement of the subway. Since there is sufficient brightness in these environments, the gray-scale image or color image can be directly processed.

[0039] The second motion state detection mode is applicable to lightless environments. In such extreme conditions, since the image is very dark, if the optical flow detection algorithm is directly used to judge the motion state of the subway, a large error will be caused. Therefore, the present invention converts the pictures of the lightless environment type into binary images, so as to detect the motion state of the subway through the pixel information in the binary images.

[0040] Based on the above embodiments, the preset brightness thresholds include a first brightness threshold and a second brightness threshold, where the first brightness threshold is greater than the second brightness threshold; Determining the type of light environment corresponding to the driving vision video image sequence according to the gray value and the preset brightness threshold includes: If the gray value is greater than the first brightness threshold, determining that the type of light environment of the driving vision video image sequence is the bright environment; If the gray value is less than or equal to the first brightness threshold and greater than the second brightness threshold, determining that the type of light environment of the driving vision video image sequence is the dim environment; If the gray value is less than or equal to the second brightness threshold, determining that the type of light environment of the driving vision video image sequence is the lightless environment.

[0041] In the present invention, two grayscale thresholds can be set, namely the first brightness threshold and the second brightness threshold. These two brightness thresholds are used to divide the light environment into three different categories. For example, the first brightness threshold is 125 and the second brightness threshold is 60.

[0042] Specifically, when the grayscale value in the picture (this grayscale value can be obtained by calculating the grayscale mean of the picture) is greater than the first brightness threshold (i.e., grayscale value > 125), it indicates that the video image is very bright and contains the most information. During the operation of the subway, such an environment usually corresponds to an outdoor scene with sufficient sunlight. Most of the pixel points in the picture are bright, and there are more points available for optical flow calculation. It is determined that the current light environment type is a bright environment.

[0043] When the grayscale value is less than or equal to the first brightness threshold (i.e., grayscale value ≤ 125) and greater than the second brightness threshold (i.e., grayscale value > 60), the video image is relatively dim, but some information is still visible. This corresponds to the scenario where the subway may be running at the tunnel entrance, at dusk, or at night with weak light. In these cases, although the number of pixel points available for optical flow calculation decreases, there is still a part that is effective. It is determined that the current light environment type is a dim environment.

[0044] When the grayscale value is less than or equal to the second brightness threshold (i.e., grayscale value ≤ 60), the video image is very dark and contains almost no effective information, corresponding to the scenario where the subway is running in a completely dark tunnel or at night without light irradiation. In such an environment, there are very few pixel points available for optical flow calculation, and most of the pixel points may be black. It is determined that the current light environment type is a lightless environment.

[0045] In practical applications, the brightness threshold can be adjusted according to the specific camera performance, environmental conditions, and the scenario of train operation. In addition, the calculation of the grayscale value in the present invention can be based on the average grayscale of the image or the grayscale statistics of a specific area to ensure the accurate and reliable judgment of the light environment type.

[0046] Based on the above embodiments, the method further includes: Cropping each frame of the driving vision video image sequence based on the correspondence between the light environment type and the preset image cropping area, to obtain the cropped driving vision video image sequence, where the ranges of the preset image cropping areas corresponding to the bright environment, the dim environment, and the lightless environment increase in turn; Detecting the motion state of the target train based on the pixel information and the target motion state detection mode between each frame of the driving vision video image sequence, to obtain the vehicle motion state detection result of the target train, including: Based on the pixel information and the target motion state detection mode between each frame of the cropped driving vision video image sequence, the motion state of the target train is detected to obtain the vehicle motion state detection result of the target train.

[0047] In the present invention, for the possible light environments during subway operation, the pictures that need to perform optical flow calculation are cropped, so that the amount of optical flow calculation for the cropped pictures is reduced, and the operation speed is improved.

[0048] Specifically, in a bright environment, the information contained in the picture is the most, and many pixel points can be used to calculate the optical flow to judge the subway motion state. In an outdoor scene (picture gray scale mean > 125), the cropping range of the picture is the largest. For example, the picture of each frame of the acquired video data is 1280×1024×3. Let the original picture be source_image, then the preset image cropping area belongs to source_image [700:990, 390:714, Start:End], so as to crop the original picture into a picture of 290×324×3 size. The area contained in the cropped picture is a small part of the railway information.

[0049] For a dim environment (i.e., 60 < picture gray scale mean ≤ 125), the information contained in the picture is reduced, most of the picture pixel points are black, and only a few pixel points can be used to calculate the optical flow to judge the subway motion state. Therefore, more of the original picture is retained after cropping. The preset image acquisition area belongs to source_image [300:800, 300:End, Star:End], and the original picture can be cropped into a picture of 500×724×3 size. The area contained in the cropped image is the railway and part of the wall.

[0050] For a basically lightless environment (picture gray scale mean ≤ 60), the information contained in the picture is the least. The preset image acquisition area belongs to source_image [150:790, 260:1010, Star:End], and the original picture can be cropped into a picture of 640×750×3 size, so as to include the light area in the picture as much as possible in the cropped picture.

[0051] Based on the above embodiments, the detecting the motion state of the target train based on the pixel information and the target motion state detection mode between each frame of the driving vision video image sequence to obtain the vehicle motion state detection result of the target train includes: When determining that the light environment type is the bright environment or the dim environment, based on the first motion state detection mode, judge the pixel displacement distance between adjacent frame images in the driving vision video image sequence; If the pixel displacement distance of the minimum displacement pixel point between the adjacent frame images is less than or equal to the minimum displacement threshold, and the pixel displacement distance of the maximum displacement pixel point between the adjacent frame images is less than or equal to the maximum displacement threshold, determine that the vehicle motion state detection result is the stationary state; If the pixel displacement distance of the maximum displacement pixel point between the adjacent frame images is greater than the maximum displacement threshold, determine that the vehicle motion state detection result is the motion state.

[0052] In the present invention, when determining that the current light environment type is the bright environment or the dim environment, the first motion state detection mode is used to analyze adjacent frame images in the driving vision video image sequence. This mode is based on optical flow calculation, and judges the motion state of the subway by comparing the displacement distance of pixel points between adjacent frame images.

[0053] Specifically, traverse all pixel points in the adjacent frame images to find the pixel point with the minimum displacement distance, and this minimum displacement distance reflects the relatively stationary part of the image. Similarly, find the pixel point with the maximum displacement distance in the adjacent frame images, and this maximum displacement distance reflects the most obvious moving part of the image. If the minimum displacement distance is less than or equal to the preset minimum displacement threshold D (in the present invention, the preset minimum displacement threshold D is 0.005), and, the maximum displacement distance is less than or equal to the preset maximum displacement threshold S (in the present invention, the preset maximum displacement threshold S is 0.8), then it can be considered that the entire image content has not moved significantly, and at this time, the motion state of the subway can be determined to be the stationary state.

[0054] If the pixel displacement distance of the maximum displacement pixel point between the adjacent frame images is greater than the preset maximum displacement threshold S, then the motion state of the subway can be determined to be the motion state.

[0055] In the present invention, the preset minimum displacement threshold D and the preset maximum displacement threshold S can be fine-tuned according to the specific rail operation city and line to adapt to different environments and conditions. Considering the problem that the detection result is inaccurate due to the train jitter caused by passengers getting on and off the train during the train motion state detection, and for the optical flow calculation during the train movement, there may also be a situation where the calculated displacement of some pixel points is relatively small, resulting in errors. The present invention uses both the maximum and minimum pixel displacements to judge the train motion state, and adds a threshold screening in the train motion prediction link, improving the accuracy of the train motion state detection.

[0056] Based on the above embodiments, the method for detecting the motion state of the target train by detecting the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode, and obtaining the vehicle motion state detection result of the target train further includes: When determining that the light environment type is the bright environment or the dim environment, based on the first motion state detection mode, judge the pixel displacement distance between consecutive multiple frames of the driving vision video image sequence; If the pixel displacement distance of all the minimum displacement pixel points between the consecutive multiple frames of images is less than or equal to the minimum displacement threshold, and the pixel displacement distance of all the maximum displacement pixel points between the consecutive multiple frames of images is less than or equal to the maximum displacement threshold, determine that the vehicle motion state detection result is the stationary state; If the pixel displacement distance of all the maximum displacement pixel points between the consecutive multiple frames of images is greater than the maximum displacement threshold, determine that the vehicle motion state detection result is the motion state.

[0057] In the present invention, in order to avoid misjudgment caused by subway jitter, equipment aging, video frame freezing, etc., 10 consecutive frames of images are continuously judged to determine the motion state of the subway. Specifically, based on the first motion state detection mode provided in the above embodiments, if the detection results of 10 consecutive frames of images are all in the motion state, it is finally determined that the subway is in the motion state. If 10 consecutive frames of images are all detected as stationary, it is finally determined that the subway is stationary. If the detection results of 10 consecutive frames of images show fluctuations (i.e., both motion and stationary), then the next 10 consecutive frames of images are rejudged until a stable judgment result is obtained, that is, all 10 frames of images are determined to be in motion or stationary.

[0058] By comprehensively considering the displacement distances of the minimum and maximum displacement pixel points between adjacent frames of images and the judgment results of consecutive multiple frames of images, the present invention can more accurately judge the motion state of the train.

[0059] Based on the above embodiments, the method for detecting the motion state of the target train by detecting the pixel information between each frame of the driving vision video image sequence and the target motion state detection mode, and obtaining the vehicle motion state detection result of the target train further includes: When determining that the light environment type is the lightless environment, based on the second motion state detection mode, judge the binarized pixel values between the first target frame image and the second target frame image in the driving vision video image sequence, where the first target frame image is the current frame image in the driving vision video image sequence, and the second target frame image is an image in the driving vision video image sequence that is at least 5 frames apart from the first target frame image; If the overlapping ratio of the binary images between the first target frame image and the second target frame image is greater than or equal to a preset overlapping ratio threshold, determine that the vehicle motion state detection result is the stationary state; If the overlapping ratio of the binary images between the first target frame image and the second target frame image is less than the preset overlapping ratio threshold, determine that the vehicle motion state detection result is the moving state; Wherein, the overlapping ratio of the binary images is calculated based on the binary pixel values between the first target frame image and the second target frame image.

[0060] In the present invention, after determining that the current is a lightless environment, the second motion state detection mode will be adopted to judge the motion state of the vehicle. Different from the motion state detection in bright or dim environments, due to the extremely scarce image information in the lightless environment, methods such as optical flow calculation may not be effectively applied. Therefore, the present invention adopts a judgment method based on the overlapping ratio of binary images.

[0061] In the present invention, first, images separated by a certain number of frames (i.e., the first target frame image and the second target frame image) are selected to avoid misjudgment caused by the slight vibration of the vehicle or the slight shaking of the camera. By comparing two frames of images separated by a certain time, the actual motion state of the vehicle can be captured more accurately.

[0062] After determining the target frame image, it is necessary to perform binary processing on the target frame image. Binary processing is the process of converting an image into an image that only contains two colors, black and white, where white represents the foreground (such as a vehicle or a road), and black represents the background (such as the darkness at night). By performing binary processing, the image information can be simplified for subsequent analysis.

[0063] Next, calculate the overlapping ratio of the binary images between the first target frame image and the second target frame image. This overlapping ratio of the binary images is calculated by comparing the pixel values at the same positions in the two frames of images. If the overlapping ratio of the binary images between the first target frame image and the second target frame image is greater than or equal to a preset overlapping ratio threshold (such as 85% or higher), it can be considered that the vehicle has hardly moved between these two frames of images, so the vehicle motion state detection result is determined to be the stationary state.

[0064] If the overlapping ratio of the binary images between the first target frame image and the second target frame image is less than the preset overlapping ratio threshold, it can be considered that the vehicle has moved significantly between these two frames of images, so the vehicle motion state detection result is determined to be the moving state.

[0065] Based on the above embodiments, the overlapping ratio of the binary images is calculated through the following steps: Perform binarization processing on the first target frame image to obtain a first binarized image; perform binarization processing on the second target frame image to obtain a second binarized image; Obtain the sum result of the first pixel points and the sum result of the second pixel point values, where the sum result of the first pixel points is the sum of the binarized pixel values of each pixel point in the first binarized image, and the sum result of the second pixel points is the sum of the binarized pixel values of each pixel point in the second binarized image; Based on the pixel coordinates in the first binarized image and the second binarized image, perform an AND operation on the pixel values corresponding to the same pixel coordinates in the first binarized image and the second binarized image in sequence to construct a third binarized image; Obtain the sum result of the third pixel points, where the sum result of the third pixel points is the sum of the binarized pixel values of each pixel point in the third binarized image; Obtain a first coincidence ratio according to the ratio between the sum result of the third pixel points and the sum result of the first pixel points; obtain a second coincidence ratio according to the ratio between the sum result of the third pixel points and the sum result of the second pixel points; Obtain the binarized image coincidence ratio according to the ratio between the first coincidence ratio and the second coincidence ratio.

[0066] In the present invention, for a lightless environment, a binarized image is used to judge the motion state of the subway. First, for the original images in the driving vision video image sequence, a binarization formula is used for processing, so as to convert them into binary images. The binarization formula is specifically: ; Wherein, src ( x , y ) represents the pixel value of the input image at the position of ( x , y ), dst ( x , y ) represents the pixel value of the output binary image at the same position, maxval represents the maximum value corresponding to binarization, threshold represents the binarization threshold. In the present invention, maxval is set to 255, threshold is set to 50.

[0067] In the present invention, the current frame image (i.e., the first target frame image) is denoted as F1, and the image five frames apart (i.e., the second target frame image) is denoted as F5. Through the above binarization formula, the images F1 and F5 are binarized to obtain two new binarized images F1_image_mask (i.e., the first binarized image) and F5_image_mask (i.e., the second binarized image). Then, using the following pixel value summation formula, the sum of the pixel values in the binarized images is calculated: ; wherein, represents the value of each pixel point in the binarized image, n is the number of pixel points. Through the above pixel value summation formula, the sum F1_image_sum (i.e., the first pixel point summation result) of the pixel values in the binarized image F1_image_mask and the sum F5_image_sum (i.e., the second pixel point summation result) of the pixel values in the binarized image F5_image_mask can be calculated.

[0068] Then, the pixel values at the same pixel coordinates in the binarized images F1_image_mask and F5_image_mask are successively subjected to an AND operation to obtain a new binarized image And_image_mask, i.e., the third binarized image. The AND operation formula is as follows: ; wherein, C i,j represents the element value (pixel value) at the C th row i and j th column of the output binary image (i.e., the third binarized image), A i,j represents the element value at the i th row j and B i,j th column of the first binarized image, i and j th column of the second binarized image.

[0069] Furthermore, through the pixel value summation formula in the above embodiment, the sum of the pixel values in the third binarized image is calculated to obtain the third pixel point summation result And_image_sum. Then, through the coincidence ratio formula, the first coincidence ratio and the second coincidence ratio are respectively calculated. The coincidence ratio formula is: ; wherein, represents the first coincidence ratio, Represents the second coincidence ratio.

[0070] In the present invention, in order to avoid detection errors caused by the jitter of the subway when it is stationary, a preset coincidence ratio threshold Th is set for judgment. When the ratio between the first coincidence ratio and the second coincidence ratio (i.e., the binarized image coincidence ratio) is greater than or equal to the preset coincidence ratio threshold Th, it can be determined that the subway is currently in a stationary state, where the preset coincidence ratio threshold Th is 0.85.

[0071] In the present invention, for different light environments, by cropping pictures and calculating optical flow, combined with pixel point displacement statistics, the motion state of the train can be effectively judged. At the same time, continuous frame image detection is used to detect the train state in a lighted environment to reduce errors. In a basically lightless environment, the binarized image processing technology is adopted. By comparing the current frame with the image five frames later, the sum of pixel point values and the coincidence ratio are calculated, so as to accurately judge the motion state of the train. This not only improves the detection speed, but also reduces the influence of the light environment on the detection accuracy. At the same time, the errors caused by train jitter and equipment aging are avoided, realizing efficient and accurate detection of the train motion state.

[0072] Compared with the existing train motion state detection methods, the present invention adopts a simple model framework, does not rely on a complex design model, and has the characteristics of fast detection speed. Moreover, in a basically lightless environment, it can accurately detect the motion state of the train without relying on other sensors, thereby reducing the equipment cost and maintenance cost. Moreover, the model scale is small, and the code only occupies 10KB of memory.

[0073] The vehicle motion state detection system based on video signals provided by the present invention will be described below. The vehicle motion state detection system based on video signals described below can be correspondingly referred to the vehicle motion state detection method based on video signals described above.

[0074] Figure 2 Is the structural schematic diagram of the vehicle motion state detection system based on video signals provided by the present invention, as Figure 2As shown in the figure, the present invention provides a vehicle motion state detection system based on video signals, including a video image acquisition module 201 and a train motion state detection module 202. Among them, the video image acquisition module 201 is used to obtain a sequence of video images of the driving vision in front of the target train during driving; the train motion state detection module 202 is used to detect the motion state of the target train based on the pixel information between each frame of images in the sequence of driving vision video images and the target motion state detection mode, and obtain the vehicle motion state detection result of the target train; among them, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the sequence of driving vision video images; the multiple motion state detection modes are constructed according to the brightness level corresponding to the light environment type.

[0075] The vehicle motion state detection system based on video signals provided by the present invention analyzes the pixel information of consecutive frame images in front of the train during driving in a light environment and a dark environment, so as to accurately obtain the motion state of the train and improve the detection speed.

[0076] The system provided by the embodiments of the present invention is used to execute the above-mentioned method embodiments. For the specific process and detailed content, please refer to the above embodiments and will not be elaborated here.

[0077] Figure 3 The structure diagram of the electronic device provided by the present invention is shown in Figure 3 As shown in the figure, the electronic device may include: a processor 301, a communication interface 302, a memory 303, and a communication bus 304. Among them, the processor 301, the communication interface 302, and the memory 303 complete mutual communication through the communication bus 304. The processor 301 can call the logical instructions in the memory 303 to execute the vehicle motion state detection method based on video signals, and the method includes: obtaining a sequence of video images of the driving vision in front of the target train during driving; detecting the motion state of the target train based on the pixel information between each frame of images in the sequence of driving vision video images and the target motion state detection mode, and obtaining the vehicle motion state detection result of the target train; among them, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the sequence of driving vision video images; the multiple motion state detection modes are constructed according to the brightness level corresponding to the light environment type.

[0078] In addition, when the logical instructions in the above-mentioned memory 303 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0079] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the vehicle motion state detection method based on video signals provided by the above-mentioned various methods. The method includes: obtaining a sequence of driving vision video images in front of a target train during driving; based on the pixel information between each frame of images in the sequence of driving vision video images and a target motion state detection mode, detecting the motion state of the target train to obtain a vehicle motion state detection result of the target train; wherein, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the sequence of driving vision video images; the multiple motion state detection modes are constructed according to the brightness corresponding to the light environment type.

[0080] In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the vehicle motion state detection method based on video signals provided by the above-mentioned various embodiments. The method includes: obtaining a sequence of driving vision video images in front of a target train during driving; based on the pixel information between each frame of images in the sequence of driving vision video images and a target motion state detection mode, detecting the motion state of the target train to obtain a vehicle motion state detection result of the target train; wherein, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the sequence of driving vision video images; the multiple motion state detection modes are constructed according to the brightness corresponding to the light environment type.

[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0082] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the motion state of a vehicle based on a video signal, characterized in that, Including: Obtain a sequence of video images of the driving vision in front of the target train during driving; Based on the pixel information between each frame of images in the sequence of video images of the driving vision and the target motion state detection mode, detect the motion state of the target train to obtain the vehicle motion state detection result of the target train; Wherein, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the sequence of video images of the driving vision; the multiple motion state detection modes are constructed according to the brightness degree corresponding to the light environment type.

2. The vehicle motion state detection method based on a video signal according to claim 1, wherein The light environment type is determined through the following steps: According to the pixel values of each frame of images in the sequence of video images of the driving vision, calculate the gray value corresponding to each frame of image; According to the gray value and the preset brightness threshold, determine the light environment type corresponding to the sequence of video images of the driving vision; Wherein, the light environment type includes a bright environment, a dim environment and a lightless environment; the multiple motion state detection modes include a first motion state detection mode and a second motion state detection mode, and the first motion state detection mode is used to detect the sequence of video images of the driving vision in the bright environment or the dim environment; the second motion state detection mode is used to detect the sequence of video images of the driving vision in the lightless environment.

3. The vehicle motion state detection method based on a video signal according to claim 2, wherein The preset brightness threshold includes a first brightness threshold and a second brightness threshold, wherein the first brightness threshold is greater than the second brightness threshold; The determining the light environment type corresponding to the sequence of video images of the driving vision according to the gray value and the preset brightness threshold includes: If the gray value is greater than the first brightness threshold, determine that the light environment type of the sequence of video images of the driving vision is the bright environment; If the gray value is less than or equal to the first brightness threshold and greater than the second brightness threshold, determine that the light environment type of the sequence of video images of the driving vision is the dim environment; If the gray value is less than or equal to the second brightness threshold, determine that the light environment type of the sequence of video images of the driving vision is the lightless environment.

4. The vehicle motion state detection method based on video signals according to claim 2, wherein The detecting the motion state of the target train based on the pixel information between each frame of images in the sequence of video images of the driving vision and the target motion state detection mode to obtain the vehicle motion state detection result of the target train includes: When determining that the light environment type is the bright environment or the dim environment, based on the first motion state detection mode, judge the pixel displacement distance between adjacent frame images in the sequence of video images of the driving vision; If the pixel displacement distance of the minimum displacement pixel point between the adjacent frame images is less than or equal to the minimum displacement threshold, and the pixel displacement distance of the maximum displacement pixel point between the adjacent frame images is less than or equal to the maximum displacement threshold, determine that the vehicle motion state detection result is the stationary state; If the pixel displacement distance of the maximum displacement pixel points between the adjacent frame images is greater than the maximum displacement threshold, determine that the vehicle motion state detection result is the motion state.

5. The vehicle motion state detection method based on a video signal according to claim 2, wherein The method for detecting the motion state of the target train based on the pixel information between each frame image in the driving vision video image sequence and the target motion state detection mode, and obtaining the vehicle motion state detection result of the target train further includes: When determining that the light environment type is the bright environment or the dim environment, based on the first motion state detection mode, judge the pixel displacement distance between consecutive multiple frame images in the driving vision video image sequence; If the pixel displacement distances of all the minimum displacement pixel points between the consecutive multiple frame images are less than or equal to the minimum displacement threshold, and the pixel displacement distances of all the maximum displacement pixel points between the consecutive multiple frame images are less than or equal to the maximum displacement threshold, determine that the vehicle motion state detection result is the stationary state; If the pixel displacement distances of all the maximum displacement pixel points between the consecutive multiple frame images are greater than the maximum displacement threshold, determine that the vehicle motion state detection result is the motion state.

6. The vehicle motion state detection method based on a video signal according to claim 2, wherein The method for detecting the motion state of the target train based on the pixel information between each frame image in the driving vision video image sequence and the target motion state detection mode, and obtaining the vehicle motion state detection result of the target train further includes: When determining that the light environment type is the lightless environment, based on the second motion state detection mode, judge the binary pixel values between the first target frame image and the second target frame image in the driving vision video image sequence, where the first target frame image is the current frame image in the driving vision video image sequence, and the second target frame image is the image in the driving vision video image sequence that is at least 5 frames apart from the first target frame image; If the binary image coincidence ratio between the first target frame image and the second target frame image is greater than or equal to the preset coincidence ratio threshold, determine that the vehicle motion state detection result is the stationary state; If the binary image coincidence ratio between the first target frame image and the second target frame image is less than the preset coincidence ratio threshold, determine that the vehicle motion state detection result is the motion state; Wherein, the binary image coincidence ratio is calculated based on the binary pixel values between the first target frame image and the second target frame image.

7. The vehicle motion state detection method based on a video signal according to claim 6, wherein The binary image coincidence ratio is calculated through the following steps: Perform binary processing on the first target frame image to obtain a first binary image; perform binary processing on the second target frame image to obtain a second binary image; Obtain the first pixel point summation result and the second pixel point value summation result, where the first pixel point summation result is the sum of the binary pixel values of each pixel point in the first binary image, and the second pixel point summation result is the sum of the binary pixel values of each pixel point in the second binary image; Based on the pixel coordinates in the first binary image and the second binary image, perform an AND operation on the pixel values corresponding to the same pixel coordinates in the first binary image and the second binary image in sequence to construct a third binary image; Obtain the sum result of the third pixel points, where the sum result of the third pixel points is the sum of the binary pixel values of each pixel point in the third binary image; Obtain a first coincidence ratio according to the ratio between the sum result of the third pixel points and the sum result of the first pixel points; obtain a second coincidence ratio according to the ratio between the sum result of the third pixel points and the sum result of the second pixel points; Obtain the binary image coincidence ratio according to the ratio between the first coincidence ratio and the second coincidence ratio.

8. The method for detecting a vehicle motion state based on a video signal according to any one of claims 2 to 7, characterized in that, The method further includes: Based on the correspondence between the light environment type and the preset image cropping area, crop each frame of the driving vision video image sequence to obtain a cropped driving vision video image sequence, where the ranges of the preset image cropping areas corresponding to the bright environment, the dim environment, and the lightless environment increase in sequence; The detecting the motion state of the target train based on the pixel information and the target motion state detection mode between each frame of the driving vision video image sequence to obtain the vehicle motion state detection result of the target train includes: Based on the pixel information and the target motion state detection mode between each frame of the cropped driving vision video image sequence, detect the motion state of the target train to obtain the vehicle motion state detection result of the target train.

9. A vehicle motion state detection system based on video signals, characterized in that, Includes: A video image acquisition module, configured to acquire a driving vision video image sequence in front of the target train during driving; A train motion state detection module, configured to detect the motion state of the target train based on the pixel information and the target motion state detection mode between each frame of the driving vision video image sequence to obtain the vehicle motion state detection result of the target train; Wherein, the target motion state detection mode is determined from multiple motion state detection modes according to the light environment type of the driving vision video image sequence; the multiple motion state detection modes are constructed according to the brightness degree corresponding to the light environment type.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the vehicle motion state detection method based on video signals according to any one of claims 1 to 8.

Images